[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:18 dist-test-slave-jlzn krb5kdc[2781](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:18 dist-test-slave-jlzn krb5kdc[2781](info): set up 2 sockets
Aug 08 15:02:18 dist-test-slave-jlzn krb5kdc[2781](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 15:02:19.395167 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:19 dist-test-slave-jlzn krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:19.556440 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35317
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:33115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:35317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:19.871493 2801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:19.871776 2801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:19.871909 2801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:19.878921 2801 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:19.879014 2801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:19.879066 2801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:19.879120 2801 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:19.879163 2801 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:19.888489 2801 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:35317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:19.889616 2801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:19.890909 2801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:19.902119 2811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:19.901988 2816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:19.902101 2812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:19.902770 2801 server_base.cc:1034] running on GCE node
I20260808 15:02:19.903584 2801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:19.904934 2801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:19.906126 2801 hybrid_clock.cc:648] HybridClock initialized: now 1786201339906097 us; error 79 us; skew 500 ppm
I20260808 15:02:19.906426 2801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:19 dist-test-slave-jlzn krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201339, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:19.909598 2801 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:19.910966 2801 webserver.cc:460] Webserver started at http://127.2.180.254:38937/ using document root <none> and password file <none>
I20260808 15:02:19.911612 2801 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:19.911728 2801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:19.912045 2801 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:19.915004 2801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "b059c1de200d4178b13fb637d0f8e8ba"
format_stamp: "Formatted at 2026-08-08 15:02:19 on dist-test-slave-jlzn"
server_key: "87aeb15fc73f83f9d1397234a37050a4"
server_key_iv: "c9a22cb3c3c179dc299ec2b336f4eb86"
server_key_version: "encryptionkey@0"
I20260808 15:02:19.915710 2801 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "b059c1de200d4178b13fb637d0f8e8ba"
format_stamp: "Formatted at 2026-08-08 15:02:19 on dist-test-slave-jlzn"
server_key: "87aeb15fc73f83f9d1397234a37050a4"
server_key_iv: "c9a22cb3c3c179dc299ec2b336f4eb86"
server_key_version: "encryptionkey@0"
I20260808 15:02:19.920542 2801 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:02:19.923743 2825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:19.924963 2801 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:02:19.925120 2801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "b059c1de200d4178b13fb637d0f8e8ba"
format_stamp: "Formatted at 2026-08-08 15:02:19 on dist-test-slave-jlzn"
server_key: "87aeb15fc73f83f9d1397234a37050a4"
server_key_iv: "c9a22cb3c3c179dc299ec2b336f4eb86"
server_key_version: "encryptionkey@0"
I20260808 15:02:19.925304 2801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:19.945844 2801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:19.949667 2801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:19.949913 2801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:19.967561 2801 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:35317
I20260808 15:02:19.967587 2891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:35317 every 8 connection(s)
I20260808 15:02:19.968847 2801 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:19.972323 2893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:19.978976 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 2801
I20260808 15:02:19.979312 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:19.981711 2771 external_mini_cluster.cc:1428] Setting key ad849b75ed15a9d3fb13581e895a7a8e
I20260808 15:02:19.983424 2893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba: Bootstrap starting.
I20260808 15:02:19.986824 2893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:19.987955 2893 log.cc:824] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:19.990671 2893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba: No bootstrap required, opened a new log
I20260808 15:02:19.998716 2893 raft_consensus.cc:348] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b059c1de200d4178b13fb637d0f8e8ba" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35317 } }
I20260808 15:02:19.999181 2893 raft_consensus.cc:374] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:19.999295 2893 raft_consensus.cc:729] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b059c1de200d4178b13fb637d0f8e8ba, State: Initialized, Role: FOLLOWER
I20260808 15:02:19.999858 2893 consensus_queue.cc:260] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b059c1de200d4178b13fb637d0f8e8ba" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35317 } }
I20260808 15:02:20.000064 2893 raft_consensus.cc:388] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:20.000196 2893 raft_consensus.cc:482] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:20.000340 2893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:20.002321 2893 raft_consensus.cc:504] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b059c1de200d4178b13fb637d0f8e8ba" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35317 } }
I20260808 15:02:20.002974 2893 leader_election.cc:302] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b059c1de200d4178b13fb637d0f8e8ba; no voters:
I20260808 15:02:20.003589 2893 leader_election.cc:288] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:20.003707 2900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:20.004757 2900 raft_consensus.cc:686] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [term 1 LEADER]: Becoming Leader. State: Replica: b059c1de200d4178b13fb637d0f8e8ba, State: Running, Role: LEADER
I20260808 15:02:20.005349 2900 consensus_queue.cc:237] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b059c1de200d4178b13fb637d0f8e8ba" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35317 } }
I20260808 15:02:20.005538 2893 sys_catalog.cc:564] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:20.008499 2901 sys_catalog.cc:455] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b059c1de200d4178b13fb637d0f8e8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b059c1de200d4178b13fb637d0f8e8ba" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35317 } } }
I20260808 15:02:20.008842 2901 sys_catalog.cc:458] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:20.009294 2903 sys_catalog.cc:455] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader b059c1de200d4178b13fb637d0f8e8ba. Latest consensus state: current_term: 1 leader_uuid: "b059c1de200d4178b13fb637d0f8e8ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b059c1de200d4178b13fb637d0f8e8ba" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35317 } } }
I20260808 15:02:20.009598 2903 sys_catalog.cc:458] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:20.009843 2911 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 08 15:02:20 dist-test-slave-jlzn krb5kdc[2781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:20.022321 2911 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:20.026552 2897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:19.986225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35454 (local address 127.2.180.254:35317)
0808 15:02:19.987164 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:19.987185 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:19.989405 (+ 2220us) server_negotiation.cc:407] Connection header received
0808 15:02:19.998192 (+ 8787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:19.998217 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:19.998612 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:19.998997 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:20.002182 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:20.004333 (+ 2151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:20.006798 (+ 2465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:20.007993 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:20.008176 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:20.015672 (+ 7496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:20.015684 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:20.015704 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:20.015759 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:20.020325 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:20.021353 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:20.021362 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:20.021371 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:20.021588 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:20.022176 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:20.022182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:20.022186 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:20.023056 (+ 870us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:20.023497 (+ 441us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:20.024783 (+ 1286us) server_negotiation.cc:299] Negotiation successful
0808 15:02:20.025229 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":378,"threads_started":1}
I20260808 15:02:20.034642 2911 catalog_manager.cc:1347] Generated new cluster ID: e865fc2734464f6da4d7c163fc5f08b1
I20260808 15:02:20.034744 2911 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:20.042634 2911 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:20.044284 2911 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:20.056703 2911 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b059c1de200d4178b13fb637d0f8e8ba: Generated new TSK 0
I20260808 15:02:20.057595 2911 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:02:20.098999 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35317
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:33115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:20.413281 2932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:20.413542 2932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:20.413643 2932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:20.420675 2932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:20.420799 2932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:20.420953 2932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:20.430496 2932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35317
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:20.431893 2932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:20.433163 2932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:20.445284 2938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:20.445372 2939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:20.445345 2943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:20.446475 2932 server_base.cc:1034] running on GCE node
I20260808 15:02:20.447023 2932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:20.447875 2932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:20.449093 2932 hybrid_clock.cc:648] HybridClock initialized: now 1786201340449035 us; error 42 us; skew 500 ppm
I20260808 15:02:20.449385 2932 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:20 dist-test-slave-jlzn krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201340, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:20.452581 2932 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:02:20.453969 2932 webserver.cc:460] Webserver started at http://127.2.180.193:33011/ using document root <none> and password file <none>
I20260808 15:02:20.454577 2932 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:20.454676 2932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:20.454931 2932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:20.457686 2932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "9c48a0795cba4d178f561bceccd67fdf"
format_stamp: "Formatted at 2026-08-08 15:02:20 on dist-test-slave-jlzn"
server_key: "d2a55985a6931c45759bb2352b8a8fad"
server_key_iv: "90cc160753d8146aea96d78ef13b155c"
server_key_version: "encryptionkey@0"
I20260808 15:02:20.458388 2932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "9c48a0795cba4d178f561bceccd67fdf"
format_stamp: "Formatted at 2026-08-08 15:02:20 on dist-test-slave-jlzn"
server_key: "d2a55985a6931c45759bb2352b8a8fad"
server_key_iv: "90cc160753d8146aea96d78ef13b155c"
server_key_version: "encryptionkey@0"
I20260808 15:02:20.463212 2932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:02:20.466392 2954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:20.467588 2932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:02:20.467729 2932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "9c48a0795cba4d178f561bceccd67fdf"
format_stamp: "Formatted at 2026-08-08 15:02:20 on dist-test-slave-jlzn"
server_key: "d2a55985a6931c45759bb2352b8a8fad"
server_key_iv: "90cc160753d8146aea96d78ef13b155c"
server_key_version: "encryptionkey@0"
I20260808 15:02:20.467926 2932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:20.493449 2932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:20.497503 2932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:20.497794 2932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:20.498679 2932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:20.499961 2932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:20.500041 2932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:20.500160 2932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:20.500216 2932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:20.530457 2932 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:41573
I20260808 15:02:20.530478 3129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:41573 every 8 connection(s)
I20260808 15:02:20.531710 2932 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:20.532761 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 2932
I20260808 15:02:20.532980 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:20.533742 2771 external_mini_cluster.cc:1428] Setting key f88f73af8cb9366f5fb1981f01a0a587
Aug 08 15:02:20 dist-test-slave-jlzn krb5kdc[2781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201340, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
I20260808 15:02:20.552119 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:20.534440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:44315 (local address 127.2.180.254:35317)
0808 15:02:20.534851 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:20.534856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:20.535861 (+ 1005us) server_negotiation.cc:407] Connection header received
0808 15:02:20.538368 (+ 2507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:20.538372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:20.538438 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:20.538529 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:20.540363 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:20.541059 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:20.542755 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:20.543157 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:20.543268 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:20.546658 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:20.546663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:20.546666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:20.546699 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:20.549184 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:20.549845 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:20.549848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:20.549851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:20.549970 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:20.550378 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:20.550381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:20.550383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:20.550639 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:20.550776 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:20.551435 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 15:02:20.551684 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":196,"threads_started":1}
I20260808 15:02:20.552937 3133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:20.534923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35317 (local address 127.2.180.193:44315)
0808 15:02:20.535691 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:20.535746 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:20.538061 (+ 2315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:20.538770 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:20.538782 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:20.539469 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:20.540218 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:20.540241 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:20.541219 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:20.541223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:20.542612 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:20.542624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:20.543369 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:20.543377 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:20.543589 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:20.544248 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:20.544270 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:20.546455 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:20.549310 (+ 2855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:20.549318 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:20.549333 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:20.549725 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:20.550069 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:20.550075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:20.550079 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:20.550267 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:20.550774 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:20.550782 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:20.551066 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 15:02:20.551316 (+ 250us) client_negotiation.cc:240] Negotiation successful
0808 15:02:20.551799 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":323,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
I20260808 15:02:20.554322 3130 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35317
I20260808 15:02:20.554605 3130 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:20.555244 3130 heartbeater.cc:511] Master 127.2.180.254:35317 requested a full tablet report, sending...
I20260808 15:02:20.557358 2847 ts_manager.cc:194] Registered new tserver with Master: 9c48a0795cba4d178f561bceccd67fdf (127.2.180.193:41573)
I20260808 15:02:20.558756 2847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:44315
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:02:20.572959 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35317
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:33115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:20.886660 3138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:20.886940 3138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:20.887081 3138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:20.894068 3138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:20.894189 3138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:20.894345 3138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:20.903563 3138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35317
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:20.904721 3138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:20.906049 3138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:20.918303 3150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:20.918303 3151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:20.918577 3154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:20.919373 3138 server_base.cc:1034] running on GCE node
I20260808 15:02:20.920002 3138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:20.920871 3138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:20.922060 3138 hybrid_clock.cc:648] HybridClock initialized: now 1786201340921994 us; error 54 us; skew 500 ppm
I20260808 15:02:20.922350 3138 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:20 dist-test-slave-jlzn krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201340, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:20.936286 3138 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:02:20.938143 3138 webserver.cc:460] Webserver started at http://127.2.180.194:38955/ using document root <none> and password file <none>
I20260808 15:02:20.939040 3138 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:20.939189 3138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:20.939555 3138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:20.943486 3138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "fb21a29778de461387d4f950b9106db6"
format_stamp: "Formatted at 2026-08-08 15:02:20 on dist-test-slave-jlzn"
server_key: "4496a4ca41eb752ada71648d961794ee"
server_key_iv: "ed76311051ab3ca1bf9160764bac67a7"
server_key_version: "encryptionkey@0"
I20260808 15:02:20.944344 3138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "fb21a29778de461387d4f950b9106db6"
format_stamp: "Formatted at 2026-08-08 15:02:20 on dist-test-slave-jlzn"
server_key: "4496a4ca41eb752ada71648d961794ee"
server_key_iv: "ed76311051ab3ca1bf9160764bac67a7"
server_key_version: "encryptionkey@0"
I20260808 15:02:20.950387 3138 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 15:02:20.955052 3163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:20.956641 3138 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:02:20.956859 3138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "fb21a29778de461387d4f950b9106db6"
format_stamp: "Formatted at 2026-08-08 15:02:20 on dist-test-slave-jlzn"
server_key: "4496a4ca41eb752ada71648d961794ee"
server_key_iv: "ed76311051ab3ca1bf9160764bac67a7"
server_key_version: "encryptionkey@0"
I20260808 15:02:20.957087 3138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:20.973140 3138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:20.977703 3138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:20.977963 3138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:20.978883 3138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:20.980185 3138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:20.980270 3138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:20.980392 3138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:20.980450 3138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:21.019089 3138 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:43817
I20260808 15:02:21.019136 3325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:43817 every 8 connection(s)
I20260808 15:02:21.020335 3138 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:21.025815 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3138
I20260808 15:02:21.026022 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:21.026471 2771 external_mini_cluster.cc:1428] Setting key 6ebc8ee06bc15f00f05b4ea7bc3dbec4
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201340, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:21.040565 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.022758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:50057 (local address 127.2.180.254:35317)
0808 15:02:21.022957 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:21.022963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:21.024100 (+ 1137us) server_negotiation.cc:407] Connection header received
0808 15:02:21.025861 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:21.025868 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:21.025960 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:21.026073 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:21.027651 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.028616 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.030228 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.030872 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.030996 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.034878 (+ 3882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:21.034885 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:21.034890 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:21.034928 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:21.037536 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.038338 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.038342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.038344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.038472 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.038958 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.038962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.038964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.039218 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:21.039327 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:21.040112 (+ 785us) server_negotiation.cc:299] Negotiation successful
0808 15:02:21.040290 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:02:21.041874 3328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.023214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35317 (local address 127.2.180.194:50057)
0808 15:02:21.023939 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:21.023977 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:21.025591 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:21.026282 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:21.026290 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:21.026811 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:21.027482 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:21.027509 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.028756 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.028760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:21.030096 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:21.030113 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.031060 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.031065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:21.031263 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.031972 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:21.032003 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:21.034660 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:21.037677 (+ 3017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:21.037685 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:21.037697 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:21.038235 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:21.038587 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:21.038592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:21.038595 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:21.038854 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:21.039338 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:21.039346 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:21.039716 (+ 370us) client_negotiation.cc:769] Sending connection context
0808 15:02:21.040043 (+ 327us) client_negotiation.cc:240] Negotiation successful
0808 15:02:21.040528 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":279,"threads_started":1}
I20260808 15:02:21.043213 3326 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35317
I20260808 15:02:21.043532 3326 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:21.044400 3326 heartbeater.cc:511] Master 127.2.180.254:35317 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
I20260808 15:02:21.045841 2847 ts_manager.cc:194] Registered new tserver with Master: fb21a29778de461387d4f950b9106db6 (127.2.180.194:43817)
I20260808 15:02:21.046628 2847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:50057
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:02:21.066453 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35317
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:33115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:21.379245 3331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:21.379503 3331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:21.379640 3331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:21.386586 3331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:21.386686 3331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:21.386842 3331 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:21.395972 3331 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35317
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:21.397104 3331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:21.398371 3331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:21.410575 3341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:21.410673 3340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:21.410632 3343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:21.411561 3331 server_base.cc:1034] running on GCE node
I20260808 15:02:21.412185 3331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:21.413030 3331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:21.414227 3331 hybrid_clock.cc:648] HybridClock initialized: now 1786201341414185 us; error 31 us; skew 500 ppm
I20260808 15:02:21.414505 3331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201341, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:21.417466 3331 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:02:21.418754 3331 webserver.cc:460] Webserver started at http://127.2.180.195:43131/ using document root <none> and password file <none>
I20260808 15:02:21.419399 3331 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:21.419516 3331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:21.419838 3331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:21.422653 3331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "6f7da5ed613b49f9a8e58a01a2e1fbe1"
format_stamp: "Formatted at 2026-08-08 15:02:21 on dist-test-slave-jlzn"
server_key: "e3f571c8388409bebca1a197b2eb9ea1"
server_key_iv: "483fd5709eb9266c3bf381a2d9a8a3e9"
server_key_version: "encryptionkey@0"
I20260808 15:02:21.423409 3331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "6f7da5ed613b49f9a8e58a01a2e1fbe1"
format_stamp: "Formatted at 2026-08-08 15:02:21 on dist-test-slave-jlzn"
server_key: "e3f571c8388409bebca1a197b2eb9ea1"
server_key_iv: "483fd5709eb9266c3bf381a2d9a8a3e9"
server_key_version: "encryptionkey@0"
I20260808 15:02:21.428226 3331 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 15:02:21.431458 3351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:21.432776 3331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:02:21.432921 3331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "6f7da5ed613b49f9a8e58a01a2e1fbe1"
format_stamp: "Formatted at 2026-08-08 15:02:21 on dist-test-slave-jlzn"
server_key: "e3f571c8388409bebca1a197b2eb9ea1"
server_key_iv: "483fd5709eb9266c3bf381a2d9a8a3e9"
server_key_version: "encryptionkey@0"
I20260808 15:02:21.433089 3331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:21.450342 3331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:21.454327 3331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:21.454588 3331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:21.455432 3331 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:21.456727 3331 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:21.456806 3331 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:21.456928 3331 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:21.456982 3331 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:21.488145 3331 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:42281
I20260808 15:02:21.488194 3519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:42281 every 8 connection(s)
I20260808 15:02:21.489367 3331 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:21.498697 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3331
I20260808 15:02:21.498867 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:21.499289 2771 external_mini_cluster.cc:1428] Setting key c9df5be212ae2394968b8bbd98c1b48b
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201341, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:21.507740 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.491743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:49277 (local address 127.2.180.254:35317)
0808 15:02:21.491940 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:21.491945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:21.493024 (+ 1079us) server_negotiation.cc:407] Connection header received
0808 15:02:21.494748 (+ 1724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:21.494753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:21.494821 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:21.494906 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:21.496316 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.497033 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.498557 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.499031 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.499153 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.502820 (+ 3667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:21.502827 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:21.502831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:21.502871 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:21.505230 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.505833 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.505836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.505839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.505950 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.506356 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.506361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.506363 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.506636 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:21.506755 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:21.507349 (+ 594us) server_negotiation.cc:299] Negotiation successful
0808 15:02:21.507501 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 15:02:21.508587 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.492179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35317 (local address 127.2.180.195:49277)
0808 15:02:21.492871 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:21.492908 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:21.494488 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:21.495097 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:21.495104 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:21.495597 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:21.496178 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:21.496196 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.497188 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.497192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:21.498394 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:21.498407 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.499255 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.499263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:21.499487 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.500202 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:21.500238 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:21.502621 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:21.505344 (+ 2723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:21.505350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:21.505360 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:21.505747 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:21.506022 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:21.506025 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:21.506027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:21.506239 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:21.506737 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:21.506742 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:21.507063 (+ 321us) client_negotiation.cc:769] Sending connection context
0808 15:02:21.507282 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 15:02:21.507616 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":273,"threads_started":1}
I20260808 15:02:21.509802 3520 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35317
I20260808 15:02:21.510041 3520 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:21.510679 3520 heartbeater.cc:511] Master 127.2.180.254:35317 requested a full tablet report, sending...
I20260808 15:02:21.512014 2847 ts_manager.cc:194] Registered new tserver with Master: 6f7da5ed613b49f9a8e58a01a2e1fbe1 (127.2.180.195:42281)
I20260808 15:02:21.512763 2847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:49277
I20260808 15:02:21.515199 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201341, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201341, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:21.560844 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.546401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44542 (local address 127.2.180.193:41573)
0808 15:02:21.547007 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:21.547015 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:21.547048 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:02:21.547120 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:21.547126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:21.547318 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:21.547448 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:21.548884 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.549894 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.551136 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.551719 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.551855 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.555480 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:21.555491 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:21.555503 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:21.555544 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:21.558172 (+ 2628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.558789 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.558796 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.558799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.558924 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.559304 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.559308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.559310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.559644 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:21.559950 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:21.560264 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 15:02:21.560525 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":291,"threads_started":1}
I20260808 15:02:21.561976 3130 heartbeater.cc:503] Master 127.2.180.254:35317 was elected leader, sending a full tablet report...
W20260808 15:02:21.562098 3002 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44542
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:21.602008 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.589444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44552 (local address 127.2.180.193:41573)
0808 15:02:21.589657 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:21.589663 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:21.589830 (+ 167us) server_negotiation.cc:407] Connection header received
0808 15:02:21.590011 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:21.590015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:21.590104 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:21.590185 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:21.591439 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.592179 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.593365 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.593800 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.593907 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.597434 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:21.597439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:21.597442 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:21.597474 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:21.599666 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.600275 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.600280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.600282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.600405 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.600737 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.600742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.600745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.600985 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:21.601204 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:21.601504 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 15:02:21.601723 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:02:21.603379 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2932
I20260808 15:02:21.614892 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.604246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44554 (local address 127.2.180.193:41573)
0808 15:02:21.604437 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:21.604442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:21.604706 (+ 264us) server_negotiation.cc:407] Connection header received
0808 15:02:21.604845 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:21.604848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:21.604901 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:21.604973 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:21.606160 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.606913 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.608141 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.608537 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.608639 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.610515 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:21.610519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:21.610522 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:21.610549 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:21.612953 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.613473 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.613476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.613478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.613612 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.613862 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.613864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.613866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.614088 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:21.614257 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:21.614421 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 15:02:21.614574 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:02:21.615523 3119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:21.695999 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 2932
I20260808 15:02:21.715785 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3138
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:02:21.731267 3539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.716711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.2.180.194:43817)
0808 15:02:21.717232 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:21.717239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:21.717263 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:02:21.717322 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:21.717329 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:21.717528 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:21.717668 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:21.718843 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.720072 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.721322 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.721883 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.722009 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.725851 (+ 3842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:21.725862 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:21.725875 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:21.725906 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:21.728536 (+ 2630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.729254 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.729264 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.729268 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.729392 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.729757 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.729760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.729762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.730101 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:21.730392 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:21.730690 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 15:02:21.730938 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":259,"threads_started":1}
I20260808 15:02:21.732570 3320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:21.814265 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3138
I20260808 15:02:21.834319 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3331
Aug 08 15:02:21 dist-test-slave-jlzn krb5kdc[2781](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:02:21.849933 3541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:21.835342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44332 (local address 127.2.180.195:42281)
0808 15:02:21.835984 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:21.835992 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:21.836021 (+ 29us) server_negotiation.cc:407] Connection header received
0808 15:02:21.836084 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:21.836090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:21.836311 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:21.836466 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:21.837813 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.838852 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.840266 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:21.840816 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:21.840929 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:21.844744 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:21.844754 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:21.844768 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:21.844804 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:21.847336 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.847961 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.847967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.847969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.848092 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:21.848439 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:21.848443 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:21.848445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:21.848777 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:21.849060 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:21.849371 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 15:02:21.849621 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":320,"threads_started":1}
I20260808 15:02:21.851147 3515 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:21.931777 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3331
I20260808 15:02:21.950110 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2801
I20260808 15:02:21.951246 2887 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:22.050561 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 2801
2026-08-08T15:02:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (3883 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:22 dist-test-slave-jlzn krb5kdc[3549](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:22 dist-test-slave-jlzn krb5kdc[3549](info): set up 2 sockets
Aug 08 15:02:22 dist-test-slave-jlzn krb5kdc[3549](info): commencing operation
krb5kdc: starting...
W20260808 15:02:23.204789 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:23 dist-test-slave-jlzn krb5kdc[3549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:23.362802 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:33829
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:33829
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:23.667668 3566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:23.667990 3566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:23.668118 3566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:23.675002 3566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:23.675092 3566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:23.675158 3566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:23.675235 3566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:23.675294 3566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:23.684579 3566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:33829
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:33829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:23.685717 3566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:23.686941 3566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:23.697851 3574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:23.697885 3575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:23.698076 3578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:23.698709 3566 server_base.cc:1034] running on GCE node
I20260808 15:02:23.699406 3566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:23.700716 3566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:23.701934 3566 hybrid_clock.cc:648] HybridClock initialized: now 1786201343701892 us; error 25 us; skew 500 ppm
I20260808 15:02:23.702251 3566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:23 dist-test-slave-jlzn krb5kdc[3549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201343, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:23.710358 3566 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:23.711781 3566 webserver.cc:460] Webserver started at http://127.2.180.254:32957/ using document root <none> and password file <none>
I20260808 15:02:23.712466 3566 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:23.712580 3566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:23.712870 3566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:23.715678 3566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "785197b6ff6f43f2a6c707be62a4ab65"
format_stamp: "Formatted at 2026-08-08 15:02:23 on dist-test-slave-jlzn"
server_key: "980104adc7f6a78d82f6cae8d8926566"
server_key_iv: "90aad17f8c3155d913c90822977af9d8"
server_key_version: "encryptionkey@0"
I20260808 15:02:23.716430 3566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "785197b6ff6f43f2a6c707be62a4ab65"
format_stamp: "Formatted at 2026-08-08 15:02:23 on dist-test-slave-jlzn"
server_key: "980104adc7f6a78d82f6cae8d8926566"
server_key_iv: "90aad17f8c3155d913c90822977af9d8"
server_key_version: "encryptionkey@0"
I20260808 15:02:23.721328 3566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 15:02:23.724534 3586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:23.725739 3566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:02:23.725898 3566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "785197b6ff6f43f2a6c707be62a4ab65"
format_stamp: "Formatted at 2026-08-08 15:02:23 on dist-test-slave-jlzn"
server_key: "980104adc7f6a78d82f6cae8d8926566"
server_key_iv: "90aad17f8c3155d913c90822977af9d8"
server_key_version: "encryptionkey@0"
I20260808 15:02:23.726068 3566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:23.746558 3566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:23.750252 3566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:23.750494 3566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:23.767076 3566 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:33829
I20260808 15:02:23.767095 3657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:33829 every 8 connection(s)
I20260808 15:02:23.768314 3566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:23.771981 3660 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:23.776407 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3566
I20260808 15:02:23.776655 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:23.777433 2771 external_mini_cluster.cc:1428] Setting key b22b2e87eddc8da7a8dce0c2f2b84f4c
I20260808 15:02:23.782859 3660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65: Bootstrap starting.
I20260808 15:02:23.786365 3660 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:23.787578 3660 log.cc:824] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:23.791357 3660 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65: No bootstrap required, opened a new log
Aug 08 15:02:23 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:23.801515 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:23.779323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39334 (local address 127.2.180.254:33829)
0808 15:02:23.780088 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:23.780098 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:23.780141 (+ 43us) server_negotiation.cc:407] Connection header received
0808 15:02:23.781893 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:23.781915 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:23.782253 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:23.782536 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:23.784280 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:23.785571 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:23.787303 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:23.788365 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:23.788549 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:23.793385 (+ 4836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:23.793395 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:23.793413 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:23.793469 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:23.797344 (+ 3875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:23.797936 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:23.797945 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:23.797952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:23.798167 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:23.798456 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:23.798461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:23.798464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:23.799148 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:23.799432 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:23.799783 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 15:02:23.800226 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":348,"threads_started":1}
I20260808 15:02:23.801359 3660 raft_consensus.cc:348] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "785197b6ff6f43f2a6c707be62a4ab65" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33829 } }
I20260808 15:02:23.801784 3660 raft_consensus.cc:374] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:23.801879 3660 raft_consensus.cc:729] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 785197b6ff6f43f2a6c707be62a4ab65, State: Initialized, Role: FOLLOWER
I20260808 15:02:23.802397 3660 consensus_queue.cc:260] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "785197b6ff6f43f2a6c707be62a4ab65" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33829 } }
I20260808 15:02:23.802629 3660 raft_consensus.cc:388] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:23.802762 3660 raft_consensus.cc:482] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:23.802940 3660 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:23.805126 3660 raft_consensus.cc:504] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "785197b6ff6f43f2a6c707be62a4ab65" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33829 } }
I20260808 15:02:23.805569 3660 leader_election.cc:302] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 785197b6ff6f43f2a6c707be62a4ab65; no voters:
I20260808 15:02:23.806119 3660 leader_election.cc:288] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:23.806236 3668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:23.806855 3668 raft_consensus.cc:686] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [term 1 LEADER]: Becoming Leader. State: Replica: 785197b6ff6f43f2a6c707be62a4ab65, State: Running, Role: LEADER
I20260808 15:02:23.807358 3668 consensus_queue.cc:237] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "785197b6ff6f43f2a6c707be62a4ab65" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33829 } }
I20260808 15:02:23.807993 3660 sys_catalog.cc:564] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:23.810339 3671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 785197b6ff6f43f2a6c707be62a4ab65. Latest consensus state: current_term: 1 leader_uuid: "785197b6ff6f43f2a6c707be62a4ab65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "785197b6ff6f43f2a6c707be62a4ab65" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33829 } } }
I20260808 15:02:23.810904 3669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "785197b6ff6f43f2a6c707be62a4ab65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "785197b6ff6f43f2a6c707be62a4ab65" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33829 } } }
I20260808 15:02:23.811257 3669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:23.811271 3671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:23.812492 3681 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:23.817315 3681 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:23.824626 3681 catalog_manager.cc:1347] Generated new cluster ID: fa6d1c27922147e8a43740b0fdfb9448
I20260808 15:02:23.824726 3681 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:23.832114 3681 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:23.833585 3681 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:23.842206 3681 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 785197b6ff6f43f2a6c707be62a4ab65: Generated new TSK 0
I20260808 15:02:23.842885 3681 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:02:23.888510 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:33829
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:24.198284 3695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:24.198554 3695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:24.198690 3695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:24.205673 3695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:24.205776 3695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:24.205930 3695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:24.215183 3695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:33829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:24.216429 3695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:24.217720 3695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:24.229969 3703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:24.229933 3705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:24.230082 3708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:24.230829 3695 server_base.cc:1034] running on GCE node
I20260808 15:02:24.231412 3695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:24.232324 3695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:24.233520 3695 hybrid_clock.cc:648] HybridClock initialized: now 1786201344233471 us; error 42 us; skew 500 ppm
I20260808 15:02:24.233804 3695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:24 dist-test-slave-jlzn krb5kdc[3549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201344, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:24.236728 3695 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:02:24.238049 3695 webserver.cc:460] Webserver started at http://127.2.180.193:35335/ using document root <none> and password file <none>
I20260808 15:02:24.238674 3695 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:24.238790 3695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:24.239089 3695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:24.241904 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "625ff0de2cf04830a028df8dc4a12202"
format_stamp: "Formatted at 2026-08-08 15:02:24 on dist-test-slave-jlzn"
server_key: "5c6b739c30b943ac8d737dade8074e05"
server_key_iv: "28a0b056be1466324fa2a7c21c1f67b5"
server_key_version: "encryptionkey@0"
I20260808 15:02:24.242672 3695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "625ff0de2cf04830a028df8dc4a12202"
format_stamp: "Formatted at 2026-08-08 15:02:24 on dist-test-slave-jlzn"
server_key: "5c6b739c30b943ac8d737dade8074e05"
server_key_iv: "28a0b056be1466324fa2a7c21c1f67b5"
server_key_version: "encryptionkey@0"
I20260808 15:02:24.247443 3695 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 15:02:24.250681 3718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:24.251912 3695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:02:24.252068 3695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "625ff0de2cf04830a028df8dc4a12202"
format_stamp: "Formatted at 2026-08-08 15:02:24 on dist-test-slave-jlzn"
server_key: "5c6b739c30b943ac8d737dade8074e05"
server_key_iv: "28a0b056be1466324fa2a7c21c1f67b5"
server_key_version: "encryptionkey@0"
I20260808 15:02:24.252250 3695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:24.264205 3695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:24.267575 3695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:24.267858 3695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:24.268700 3695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:24.269919 3695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:24.269996 3695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:24.270119 3695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:24.270177 3695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:24.299633 3695 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:35337
I20260808 15:02:24.299681 3887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:35337 every 8 connection(s)
I20260808 15:02:24.300855 3695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:24.302399 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3695
I20260808 15:02:24.302615 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:24.303259 2771 external_mini_cluster.cc:1428] Setting key 764159b61a936986a7595787c22d642f
Aug 08 15:02:24 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201344, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:24.322218 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:24.303391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:55731 (local address 127.2.180.254:33829)
0808 15:02:24.303885 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:24.303892 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:24.304765 (+ 873us) server_negotiation.cc:407] Connection header received
0808 15:02:24.307083 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:24.307087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:24.307156 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:24.307242 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:24.309166 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:24.309971 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:24.311980 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:24.312618 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:24.312742 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:24.316472 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:24.316477 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:24.316480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:24.316511 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:24.319013 (+ 2502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:24.319879 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:24.319882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:24.319884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:24.319995 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:24.320498 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:24.320502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:24.320505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:24.320850 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:24.320971 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:24.321783 (+ 812us) server_negotiation.cc:299] Negotiation successful
0808 15:02:24.321945 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":226,"threads_started":1}
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
I20260808 15:02:24.323354 3894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:24.303934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:33829 (local address 127.2.180.193:55731)
0808 15:02:24.304628 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:24.304667 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:24.306795 (+ 2128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:24.307489 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:24.307500 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:24.308274 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:24.309001 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:24.309024 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:24.310118 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:24.310123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:24.311837 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:24.311852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:24.312813 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:24.312818 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:24.313078 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:24.313718 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:24.313745 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:24.316251 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:24.319171 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:24.319180 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:24.319195 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:24.319748 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:24.320090 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:24.320095 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:24.320098 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:24.320345 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:24.320990 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:24.320999 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:24.321411 (+ 412us) client_negotiation.cc:769] Sending connection context
0808 15:02:24.321709 (+ 298us) client_negotiation.cc:240] Negotiation successful
0808 15:02:24.322217 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":282,"threads_started":1}
I20260808 15:02:24.324921 3888 heartbeater.cc:347] Connected to a master server at 127.2.180.254:33829
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
I20260808 15:02:24.325248 3888 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:24.326082 3888 heartbeater.cc:511] Master 127.2.180.254:33829 requested a full tablet report, sending...
I20260808 15:02:24.328213 3612 ts_manager.cc:194] Registered new tserver with Master: 625ff0de2cf04830a028df8dc4a12202 (127.2.180.193:35337)
I20260808 15:02:24.330005 3612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:55731
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:02:24.344901 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:33829
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:24.650367 3898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:24.650631 3898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:24.650774 3898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:24.657810 3898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:24.657918 3898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:24.658056 3898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:24.667155 3898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:33829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:24.668344 3898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:24.669584 3898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:24.681310 3907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:24.681353 3911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:24.681460 3908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:24.682897 3898 server_base.cc:1034] running on GCE node
I20260808 15:02:24.683459 3898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:24.684329 3898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:24.685518 3898 hybrid_clock.cc:648] HybridClock initialized: now 1786201344685461 us; error 51 us; skew 500 ppm
I20260808 15:02:24.685796 3898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:24 dist-test-slave-jlzn krb5kdc[3549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201344, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:24.688616 3898 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:02:24.689939 3898 webserver.cc:460] Webserver started at http://127.2.180.194:39577/ using document root <none> and password file <none>
I20260808 15:02:24.690565 3898 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:24.690686 3898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:24.690968 3898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:24.693732 3898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "512ac475b83c4b1f9b0c617acacd8a89"
format_stamp: "Formatted at 2026-08-08 15:02:24 on dist-test-slave-jlzn"
server_key: "a1d7fd2cc71c298556b68e6b2c49e1ee"
server_key_iv: "807181c512fc63f899013952c84a3a49"
server_key_version: "encryptionkey@0"
I20260808 15:02:24.694454 3898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "512ac475b83c4b1f9b0c617acacd8a89"
format_stamp: "Formatted at 2026-08-08 15:02:24 on dist-test-slave-jlzn"
server_key: "a1d7fd2cc71c298556b68e6b2c49e1ee"
server_key_iv: "807181c512fc63f899013952c84a3a49"
server_key_version: "encryptionkey@0"
I20260808 15:02:24.699044 3898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 15:02:24.702224 3922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:24.703380 3898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:02:24.703521 3898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "512ac475b83c4b1f9b0c617acacd8a89"
format_stamp: "Formatted at 2026-08-08 15:02:24 on dist-test-slave-jlzn"
server_key: "a1d7fd2cc71c298556b68e6b2c49e1ee"
server_key_iv: "807181c512fc63f899013952c84a3a49"
server_key_version: "encryptionkey@0"
I20260808 15:02:24.703701 3898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:24.720281 3898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:24.723599 3898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:24.723870 3898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:24.724704 3898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:24.725893 3898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:24.725984 3898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:24.726101 3898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:24.726152 3898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:24.755779 3898 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:35047
I20260808 15:02:24.755860 4081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:35047 every 8 connection(s)
I20260808 15:02:24.756989 3898 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:24.758325 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3898
I20260808 15:02:24.758502 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:24.758908 2771 external_mini_cluster.cc:1428] Setting key 8bfdd706ed3603af7c9ca4410663cbc4
Aug 08 15:02:24 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201344, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
I20260808 15:02:24.778923 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:24.759286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:60023 (local address 127.2.180.254:33829)
0808 15:02:24.759508 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:24.759516 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:24.760817 (+ 1301us) server_negotiation.cc:407] Connection header received
0808 15:02:24.762611 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:24.762616 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:24.762702 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:24.762794 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:24.764190 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:24.765132 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:24.767022 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:24.767682 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:24.767835 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:24.771953 (+ 4118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:24.771960 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:24.771965 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:24.772005 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:24.775373 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:24.776299 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:24.776305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:24.776308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:24.776490 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:24.776978 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:24.776983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:24.776986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:24.777331 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:24.777585 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:24.778311 (+ 726us) server_negotiation.cc:299] Negotiation successful
0808 15:02:24.778666 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 15:02:24.779868 4086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:24.759866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:33829 (local address 127.2.180.194:60023)
0808 15:02:24.760662 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:24.760717 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:24.762357 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:24.762954 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:24.762962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:24.763454 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:24.764033 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:24.764052 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:24.765283 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:24.765286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:24.766871 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:24.766887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:24.767944 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:24.767952 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:24.768222 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:24.768933 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:24.768959 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:24.771731 (+ 2772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:24.775525 (+ 3794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:24.775533 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:24.775547 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:24.776140 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:24.776620 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:24.776625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:24.776629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:24.776844 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:24.777460 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:24.777467 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:24.777857 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 15:02:24.778148 (+ 291us) client_negotiation.cc:240] Negotiation successful
0808 15:02:24.778626 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":294,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
I20260808 15:02:24.781286 4082 heartbeater.cc:347] Connected to a master server at 127.2.180.254:33829
I20260808 15:02:24.781629 4082 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:24.782492 4082 heartbeater.cc:511] Master 127.2.180.254:33829 requested a full tablet report, sending...
I20260808 15:02:24.784107 3612 ts_manager.cc:194] Registered new tserver with Master: 512ac475b83c4b1f9b0c617acacd8a89 (127.2.180.194:35047)
I20260808 15:02:24.784873 3612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:60023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:02:24.800823 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:33829
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:35599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:25.108355 4089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:25.108609 4089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:25.108737 4089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:25.115715 4089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:25.115826 4089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:25.115958 4089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:25.125332 4089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:35599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:33829
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:25.126421 4089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:25.127599 4089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:25.139503 4099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:25.139467 4104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:25.139531 4102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:25.140156 4089 server_base.cc:1034] running on GCE node
I20260808 15:02:25.140676 4089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:25.141460 4089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:25.142642 4089 hybrid_clock.cc:648] HybridClock initialized: now 1786201345142594 us; error 42 us; skew 500 ppm
I20260808 15:02:25.142905 4089 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:25 dist-test-slave-jlzn krb5kdc[3549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201345, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:25.145810 4089 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:02:25.147046 4089 webserver.cc:460] Webserver started at http://127.2.180.195:46569/ using document root <none> and password file <none>
I20260808 15:02:25.147634 4089 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:25.147727 4089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:25.148006 4089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:25.150633 4089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "ea88c4a6e87f4b2b83340491467baf2c"
format_stamp: "Formatted at 2026-08-08 15:02:25 on dist-test-slave-jlzn"
server_key: "88b61d35bea7d3e99601af5d3a51e05a"
server_key_iv: "a8acc71548e0bbc05637d9c0b479300e"
server_key_version: "encryptionkey@0"
I20260808 15:02:25.151439 4089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "ea88c4a6e87f4b2b83340491467baf2c"
format_stamp: "Formatted at 2026-08-08 15:02:25 on dist-test-slave-jlzn"
server_key: "88b61d35bea7d3e99601af5d3a51e05a"
server_key_iv: "a8acc71548e0bbc05637d9c0b479300e"
server_key_version: "encryptionkey@0"
I20260808 15:02:25.156183 4089 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 15:02:25.159253 4115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:25.160463 4089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:02:25.160602 4089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "ea88c4a6e87f4b2b83340491467baf2c"
format_stamp: "Formatted at 2026-08-08 15:02:25 on dist-test-slave-jlzn"
server_key: "88b61d35bea7d3e99601af5d3a51e05a"
server_key_iv: "a8acc71548e0bbc05637d9c0b479300e"
server_key_version: "encryptionkey@0"
I20260808 15:02:25.160771 4089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:25.177189 4089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:25.181107 4089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:25.181360 4089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:25.182200 4089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:25.183409 4089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:25.183480 4089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:25.183588 4089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:25.183637 4089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:25.213385 4089 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:44543
I20260808 15:02:25.213425 4283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:44543 every 8 connection(s)
I20260808 15:02:25.214569 4089 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:25.224582 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 4089
I20260808 15:02:25.224750 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:25.225155 2771 external_mini_cluster.cc:1428] Setting key a29c371f948df9c3bc2b8577107bca70
Aug 08 15:02:25 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201345, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:25.232591 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.216837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:46217 (local address 127.2.180.254:33829)
0808 15:02:25.217006 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.217011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.218094 (+ 1083us) server_negotiation.cc:407] Connection header received
0808 15:02:25.219851 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.219857 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.219935 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.220021 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:25.221430 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.222158 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.223656 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.224103 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.224214 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.227660 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:25.227666 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:25.227670 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:25.227708 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:25.230146 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.230772 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.230776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.230778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.230912 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.231240 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.231243 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.231245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.231476 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:25.231588 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.232199 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.232352 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:02:25.233428 4287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.217281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:33829 (local address 127.2.180.195:46217)
0808 15:02:25.217944 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:25.217979 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:25.219576 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:25.220259 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:25.220267 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:25.220745 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:25.221269 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.221286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.222317 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.222321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.223514 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.223526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.224323 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.224331 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.224540 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.225139 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:25.225173 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:25.227448 (+ 2275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:25.230274 (+ 2826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:25.230280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:25.230290 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:25.230693 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:25.230994 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:25.230997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:25.230999 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:25.231169 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:25.231577 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:25.231582 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:25.231909 (+ 327us) client_negotiation.cc:769] Sending connection context
0808 15:02:25.232118 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 15:02:25.232455 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":260,"threads_started":1}
I20260808 15:02:25.234650 4285 heartbeater.cc:347] Connected to a master server at 127.2.180.254:33829
I20260808 15:02:25.234905 4285 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:25.235543 4285 heartbeater.cc:511] Master 127.2.180.254:33829 requested a full tablet report, sending...
I20260808 15:02:25.236845 3612 ts_manager.cc:194] Registered new tserver with Master: ea88c4a6e87f4b2b83340491467baf2c (127.2.180.195:44543)
I20260808 15:02:25.237591 3612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:46217
I20260808 15:02:25.240222 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:02:25.257477 3890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.245923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39350 (local address 127.2.180.254:33829)
0808 15:02:25.246062 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.246066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.246536 (+ 470us) server_negotiation.cc:407] Connection header received
0808 15:02:25.246688 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.246693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.246748 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.246823 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:25.248256 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.248947 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.250179 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.250580 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.250695 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.252852 (+ 2157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:25.252856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:25.252860 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:25.252888 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:25.254989 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.255589 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.255594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.255598 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.255727 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.256086 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.256091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.256094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.256457 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:25.256688 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.257022 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.257219 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:02:25.262853 3612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:02:25.269433 3612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:25.302860 4310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.280554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:35337 (local address 127.0.0.1:49254)
0808 15:02:25.294555 (+ 14001us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:25.294575 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:25.294704 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:25.295470 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:25.295477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:25.295509 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:25.295925 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.295939 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.297681 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.297687 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.299998 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.300017 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.301931 (+ 1914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.301937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.302145 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.302211 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 15:02:25.302318 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 15:02:25.302485 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13863,"mutex_wait_us":1,"thread_start_us":16277,"threads_started":1}
I20260808 15:02:25.304380 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.282014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49254 (local address 127.2.180.193:35337)
0808 15:02:25.294855 (+ 12841us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.294865 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.294896 (+ 31us) server_negotiation.cc:407] Connection header received
0808 15:02:25.295006 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.295015 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.295249 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.295446 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:25.296113 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.297544 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.300207 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.301692 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.303575 (+ 1883us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.303793 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.303884 (+ 91us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.304036 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12692,"thread_start_us":12551,"threads_started":1}
I20260808 15:02:25.305280 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.281984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57500 (local address 127.2.180.194:35047)
0808 15:02:25.295021 (+ 13037us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.295029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.297335 (+ 2306us) server_negotiation.cc:407] Connection header received
0808 15:02:25.297504 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.297511 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.297717 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.297874 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:25.298859 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.300004 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.302418 (+ 2414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.303980 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.304085 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.304261 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.304826 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.304972 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12030,"thread_start_us":12796,"threads_started":1}
I20260808 15:02:25.305485 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.280369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:44543 (local address 127.0.0.1:36470)
0808 15:02:25.281237 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:25.281283 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:25.281442 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:25.296367 (+ 14925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:25.296372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:25.296394 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:25.296839 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.296853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.298732 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.298737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.301970 (+ 3233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.301985 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.303714 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.304756 (+ 1042us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.304911 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.304942 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:02:25.305002 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 15:02:25.305117 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":1,"thread_start_us":505,"threads_started":1}
I20260808 15:02:25.306355 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.280649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36470 (local address 127.2.180.195:44543)
0808 15:02:25.295880 (+ 15231us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.295887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.295919 (+ 32us) server_negotiation.cc:407] Connection header received
0808 15:02:25.296009 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.296015 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.296209 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.296362 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:25.297079 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.298557 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.302155 (+ 3598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.303501 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.305605 (+ 2104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.305822 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.305884 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.306027 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15042,"thread_start_us":15890,"threads_started":1}
I20260808 15:02:25.306490 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.281582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:35047 (local address 127.0.0.1:57500)
0808 15:02:25.297221 (+ 15639us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:25.297238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:25.297356 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:25.298150 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:25.298155 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:25.298184 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:25.298667 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.298680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.300172 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.300181 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.302251 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.302265 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.304444 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.304450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.304611 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.304642 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:02:25.306015 (+ 1373us) client_negotiation.cc:240] Negotiation successful
0808 15:02:25.306134 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15503,"mutex_wait_us":22,"thread_start_us":12710,"threads_started":1}
I20260808 15:02:25.308314 3785 tablet_service.cc:1467] Processing CreateTablet for tablet 371c5e6ea8954840a66f3a186911ccd1 (DEFAULT_TABLE table=test-table [id=fb6d28907431451c930a19d29b70f9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:25.309809 3785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 371c5e6ea8954840a66f3a186911ccd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:25.309559 3989 tablet_service.cc:1467] Processing CreateTablet for tablet 371c5e6ea8954840a66f3a186911ccd1 (DEFAULT_TABLE table=test-table [id=fb6d28907431451c930a19d29b70f9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:25.309798 4182 tablet_service.cc:1467] Processing CreateTablet for tablet 371c5e6ea8954840a66f3a186911ccd1 (DEFAULT_TABLE table=test-table [id=fb6d28907431451c930a19d29b70f9b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:25.310726 3989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 371c5e6ea8954840a66f3a186911ccd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:25.311131 4182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 371c5e6ea8954840a66f3a186911ccd1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:25.321092 4314 tablet_bootstrap.cc:492] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89: Bootstrap starting.
I20260808 15:02:25.323688 4315 tablet_bootstrap.cc:492] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202: Bootstrap starting.
I20260808 15:02:25.324606 4314 tablet_bootstrap.cc:654] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:25.325223 4316 tablet_bootstrap.cc:492] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c: Bootstrap starting.
I20260808 15:02:25.325906 4314 log.cc:824] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:25.326988 4315 tablet_bootstrap.cc:654] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:25.328045 4316 tablet_bootstrap.cc:654] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:25.328181 4315 log.cc:824] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:25.328999 4314 tablet_bootstrap.cc:492] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89: No bootstrap required, opened a new log
I20260808 15:02:25.329066 4316 log.cc:824] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:25.329244 4314 ts_tablet_manager.cc:1397] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 15:02:25.331198 4315 tablet_bootstrap.cc:492] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202: No bootstrap required, opened a new log
I20260808 15:02:25.331449 4315 ts_tablet_manager.cc:1397] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 15:02:25.331849 4316 tablet_bootstrap.cc:492] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c: No bootstrap required, opened a new log
I20260808 15:02:25.332070 4316 ts_tablet_manager.cc:1397] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:02:25.333269 3888 heartbeater.cc:503] Master 127.2.180.254:33829 was elected leader, sending a full tablet report...
I20260808 15:02:25.340116 4315 raft_consensus.cc:348] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.340442 4315 raft_consensus.cc:374] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:25.340533 4315 raft_consensus.cc:729] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 625ff0de2cf04830a028df8dc4a12202, State: Initialized, Role: FOLLOWER
I20260808 15:02:25.341035 4315 consensus_queue.cc:260] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.341017 4314 raft_consensus.cc:348] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.341430 4314 raft_consensus.cc:374] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:25.341540 4314 raft_consensus.cc:729] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 512ac475b83c4b1f9b0c617acacd8a89, State: Initialized, Role: FOLLOWER
I20260808 15:02:25.341706 4316 raft_consensus.cc:348] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.342087 4316 raft_consensus.cc:374] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:25.342190 4316 raft_consensus.cc:729] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea88c4a6e87f4b2b83340491467baf2c, State: Initialized, Role: FOLLOWER
I20260808 15:02:25.342139 4314 consensus_queue.cc:260] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.342815 4315 ts_tablet_manager.cc:1428] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260808 15:02:25.342820 4316 consensus_queue.cc:260] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.345504 4314 ts_tablet_manager.cc:1428] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
I20260808 15:02:25.345516 4285 heartbeater.cc:503] Master 127.2.180.254:33829 was elected leader, sending a full tablet report...
I20260808 15:02:25.345963 4316 ts_tablet_manager.cc:1428] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260808 15:02:25.346777 4082 heartbeater.cc:503] Master 127.2.180.254:33829 was elected leader, sending a full tablet report...
W20260808 15:02:25.466144 4286 tablet.cc:2369] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:02:25.508850 4085 tablet.cc:2369] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:02:25.553042 3889 tablet.cc:2369] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:02:25.560025 4322 raft_consensus.cc:482] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:25.560436 4322 raft_consensus.cc:504] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.562282 4322 leader_election.cc:288] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ea88c4a6e87f4b2b83340491467baf2c (127.2.180.195:44543), 625ff0de2cf04830a028df8dc4a12202 (127.2.180.193:35337)
I20260808 15:02:25.569139 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.562740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:36499 (local address 127.2.180.193:35337)
0808 15:02:25.562937 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.562944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.563773 (+ 829us) server_negotiation.cc:407] Connection header received
0808 15:02:25.563901 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.563906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.563971 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.564090 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:25.564724 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.565471 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.567412 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.568266 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.568413 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.568484 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.568783 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.568900 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 15:02:25.569278 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.562740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:40531 (local address 127.2.180.195:44543)
0808 15:02:25.562937 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.562944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.563894 (+ 950us) server_negotiation.cc:407] Connection header received
0808 15:02:25.563968 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.563971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.564061 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.564146 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:25.564793 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.565558 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.567403 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.568202 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.568353 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.568428 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.568899 (+ 471us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.569036 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:02:25.569344 4330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.562746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:35337 (local address 127.2.180.194:36499)
0808 15:02:25.563563 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:25.563576 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:25.563729 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:25.564106 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:25.564111 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:25.564137 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:25.564564 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.564575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.565597 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.565602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.567259 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.567272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.568479 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.568485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.568634 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.568657 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:02:25.568738 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 15:02:25.568857 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":60,"spinlock_wait_cycles":32640,"thread_start_us":430,"threads_started":1}
I20260808 15:02:25.569425 4325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.562717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:44543 (local address 127.2.180.194:40531)
0808 15:02:25.563550 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:25.563572 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:25.563765 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:25.564179 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:25.564183 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:25.564214 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:25.564649 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.564658 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.565695 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.565699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.567255 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:25.567268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.568428 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.568435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:25.568592 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.568743 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 15:02:25.568842 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 15:02:25.568961 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":101,"spinlock_wait_cycles":22912,"thread_start_us":426,"threads_started":1}
I20260808 15:02:25.570469 4218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371c5e6ea8954840a66f3a186911ccd1" candidate_uuid: "512ac475b83c4b1f9b0c617acacd8a89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea88c4a6e87f4b2b83340491467baf2c" is_pre_election: true
I20260808 15:02:25.570649 3822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371c5e6ea8954840a66f3a186911ccd1" candidate_uuid: "512ac475b83c4b1f9b0c617acacd8a89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "625ff0de2cf04830a028df8dc4a12202" is_pre_election: true
I20260808 15:02:25.571085 4218 raft_consensus.cc:2454] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512ac475b83c4b1f9b0c617acacd8a89 in term 0.
I20260808 15:02:25.571132 3822 raft_consensus.cc:2454] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 512ac475b83c4b1f9b0c617acacd8a89 in term 0.
I20260808 15:02:25.571776 3926 leader_election.cc:302] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 512ac475b83c4b1f9b0c617acacd8a89, 625ff0de2cf04830a028df8dc4a12202; no voters:
I20260808 15:02:25.572204 4322 raft_consensus.cc:2781] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:25.572368 4322 raft_consensus.cc:482] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:25.572468 4322 raft_consensus.cc:3037] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:25.574558 4322 raft_consensus.cc:504] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.575062 4322 leader_election.cc:288] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [CANDIDATE]: Term 1 election: Requested vote from peers ea88c4a6e87f4b2b83340491467baf2c (127.2.180.195:44543), 625ff0de2cf04830a028df8dc4a12202 (127.2.180.193:35337)
I20260808 15:02:25.575464 4218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371c5e6ea8954840a66f3a186911ccd1" candidate_uuid: "512ac475b83c4b1f9b0c617acacd8a89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea88c4a6e87f4b2b83340491467baf2c"
I20260808 15:02:25.575521 3822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "371c5e6ea8954840a66f3a186911ccd1" candidate_uuid: "512ac475b83c4b1f9b0c617acacd8a89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "625ff0de2cf04830a028df8dc4a12202"
I20260808 15:02:25.575650 4218 raft_consensus.cc:3037] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:25.575695 3822 raft_consensus.cc:3037] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:25.577788 4218 raft_consensus.cc:2454] T 371c5e6ea8954840a66f3a186911ccd1 P ea88c4a6e87f4b2b83340491467baf2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512ac475b83c4b1f9b0c617acacd8a89 in term 1.
I20260808 15:02:25.577805 3822 raft_consensus.cc:2454] T 371c5e6ea8954840a66f3a186911ccd1 P 625ff0de2cf04830a028df8dc4a12202 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 512ac475b83c4b1f9b0c617acacd8a89 in term 1.
I20260808 15:02:25.578151 3923 leader_election.cc:302] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 512ac475b83c4b1f9b0c617acacd8a89, ea88c4a6e87f4b2b83340491467baf2c; no voters:
I20260808 15:02:25.578424 4322 raft_consensus.cc:2781] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:25.578805 4322 raft_consensus.cc:686] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [term 1 LEADER]: Becoming Leader. State: Replica: 512ac475b83c4b1f9b0c617acacd8a89, State: Running, Role: LEADER
I20260808 15:02:25.579237 4322 consensus_queue.cc:237] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } }
I20260808 15:02:25.583827 3611 catalog_manager.cc:5495] T 371c5e6ea8954840a66f3a186911ccd1 P 512ac475b83c4b1f9b0c617acacd8a89 reported cstate change: term changed from 0 to 1, leader changed from <none> to 512ac475b83c4b1f9b0c617acacd8a89 (127.2.180.194). New cstate: current_term: 1 leader_uuid: "512ac475b83c4b1f9b0c617acacd8a89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea88c4a6e87f4b2b83340491467baf2c" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "625ff0de2cf04830a028df8dc4a12202" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 35337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "512ac475b83c4b1f9b0c617acacd8a89" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35047 } health_report { overall_health: HEALTHY } } }
Aug 08 15:02:25 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201343, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:25.658296 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.644467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.2.180.193:35337)
0808 15:02:25.644659 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.644666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.645107 (+ 441us) server_negotiation.cc:407] Connection header received
0808 15:02:25.645306 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.645310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.645376 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.645460 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:25.646943 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.647701 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.648943 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.649348 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.649457 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.653242 (+ 3785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:25.653256 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:25.653272 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:25.653313 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:25.656027 (+ 2714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.656704 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.656710 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.656713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.656850 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.657137 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.657140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.657142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.657478 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:25.657724 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.657884 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.658041 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 15:02:25 dist-test-slave-jlzn krb5kdc[3549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201345, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:02:25 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201345, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:25.702781 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.689607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.2.180.193:35337)
0808 15:02:25.689789 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.689796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.690136 (+ 340us) server_negotiation.cc:407] Connection header received
0808 15:02:25.690310 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.690314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.690374 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.690453 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:25.691880 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.692648 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.693907 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.694323 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.694434 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.698016 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:25.698021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:25.698024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:25.698057 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:25.700403 (+ 2346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.700990 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.700995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.700997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.701140 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.701505 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.701509 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.701511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.701768 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:25.702002 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.702273 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.702478 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 15:02:25 dist-test-slave-jlzn krb5kdc[3549](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:02:25 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:25.745206 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.732249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49284 (local address 127.2.180.193:35337)
0808 15:02:25.732435 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.732441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.732743 (+ 302us) server_negotiation.cc:407] Connection header received
0808 15:02:25.732909 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.732913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.732974 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.733056 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:25.734379 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.735068 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.736287 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.736723 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.736835 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.740384 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:25.740389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:25.740392 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:25.740423 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:25.742689 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.743320 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.743324 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.743326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.743446 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.743791 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.743797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.743799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.744126 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:25.744361 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.744665 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.744891 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 15:02:25.746980 3760 tablet_service.cc:2908] Rejecting scan request for tablet 371c5e6ea8954840a66f3a186911ccd1: Uninitialized: clean time has not yet been initialized
I20260808 15:02:25.749922 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3695
I20260808 15:02:25.762871 4313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.750981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49300 (local address 127.2.180.193:35337)
0808 15:02:25.751115 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.751119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.751450 (+ 331us) server_negotiation.cc:407] Connection header received
0808 15:02:25.751680 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.751684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.751763 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.751901 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:25.752959 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.753607 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.754701 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.755126 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.755280 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.757970 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:25.757975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:25.757978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:25.758007 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:25.760500 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.761204 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.761208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.761210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.761318 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.761711 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.761715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.761717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.762014 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:25.762234 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.762470 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.762644 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:02:25.763450 3882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:25.855154 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3695
I20260808 15:02:25.879308 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3898
Aug 08 15:02:25 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:02:25.893163 4355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:25.880083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57504 (local address 127.2.180.194:35047)
0808 15:02:25.880573 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:25.880578 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:25.880593 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:02:25.880646 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:25.880662 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:25.880723 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:25.880813 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:25.882004 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.882718 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.884006 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:25.884404 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:25.884515 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:25.888266 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:25.888280 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:25.888295 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:25.888333 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:25.890970 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.891526 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.891536 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.891540 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.891678 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:25.891982 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:25.891985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:25.891987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:25.892341 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:25.892558 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:25.892739 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 15:02:25.892896 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":233,"threads_started":1}
I20260808 15:02:25.893805 4076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:25.980542 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3898
I20260808 15:02:26.000391 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4089
Aug 08 15:02:26 dist-test-slave-jlzn krb5kdc[3549](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201345, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:02:26.014367 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:26.001132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.2.180.195:44543)
0808 15:02:26.001308 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:26.001314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:26.001335 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:02:26.001521 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:26.001526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:26.001597 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:26.001692 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:26.002922 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:26.003645 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:26.004935 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:26.005329 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:26.005439 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:26.009405 (+ 3966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:26.009418 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:26.009434 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:26.009466 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:26.012105 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:26.012722 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:26.012728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:26.012731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:26.012866 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:26.013168 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:26.013171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:26.013172 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:26.013512 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:26.013725 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:26.013941 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 15:02:26.014098 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:02:26.014993 4279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:26.098925 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 4089
I20260808 15:02:26.118798 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3566
I20260808 15:02:26.119949 3653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:26.224177 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3566
2026-08-08T15:02:26Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4172 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:26 dist-test-slave-jlzn krb5kdc[4364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:26 dist-test-slave-jlzn krb5kdc[4364](info): set up 2 sockets
Aug 08 15:02:26 dist-test-slave-jlzn krb5kdc[4364](info): commencing operation
krb5kdc: starting...
W20260808 15:02:27.383901 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.002s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:27 dist-test-slave-jlzn krb5kdc[4364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:27.550807 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:34003
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:34003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:27.853979 4382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:27.854241 4382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:27.854370 4382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:27.861176 4382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:27.861268 4382 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 15:02:27.861334 4382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:27.861409 4382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:27.861469 4382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:27.861555 4382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:27.870841 4382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:34003
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:34003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:27.871990 4382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:27.873203 4382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:27.885385 4388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:27.885409 4390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:27.886394 4387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:27.887187 4382 server_base.cc:1034] running on GCE node
I20260808 15:02:27.888049 4382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:27.889423 4382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:27.890652 4382 hybrid_clock.cc:648] HybridClock initialized: now 1786201347890573 us; error 41 us; skew 500 ppm
I20260808 15:02:27.890967 4382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:27 dist-test-slave-jlzn krb5kdc[4364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:27.893950 4382 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:27.895391 4382 webserver.cc:460] Webserver started at http://127.2.180.254:45849/ using document root <none> and password file <none>
I20260808 15:02:27.896086 4382 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:27.896188 4382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:27.896486 4382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:27.899225 4382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "0c9ff7104f524d64b169c939d9fb48de"
format_stamp: "Formatted at 2026-08-08 15:02:27 on dist-test-slave-jlzn"
server_key: "7a67b3124889f8838a51b7fd1fc63449"
server_key_iv: "ba7e32777e9c9d1a8bede6da1fdbb4bd"
server_key_version: "encryptionkey@0"
I20260808 15:02:27.900007 4382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "0c9ff7104f524d64b169c939d9fb48de"
format_stamp: "Formatted at 2026-08-08 15:02:27 on dist-test-slave-jlzn"
server_key: "7a67b3124889f8838a51b7fd1fc63449"
server_key_iv: "ba7e32777e9c9d1a8bede6da1fdbb4bd"
server_key_version: "encryptionkey@0"
I20260808 15:02:27.904678 4382 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 15:02:27.907709 4403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:27.908885 4382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:02:27.909036 4382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "0c9ff7104f524d64b169c939d9fb48de"
format_stamp: "Formatted at 2026-08-08 15:02:27 on dist-test-slave-jlzn"
server_key: "7a67b3124889f8838a51b7fd1fc63449"
server_key_iv: "ba7e32777e9c9d1a8bede6da1fdbb4bd"
server_key_version: "encryptionkey@0"
I20260808 15:02:27.909221 4382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:27.930843 4382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:27.934358 4382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:27.934579 4382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:27.953972 4382 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:34003
I20260808 15:02:27.954005 4491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:34003 every 8 connection(s)
I20260808 15:02:27.955147 4382 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:27.958549 4492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:27.963500 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 4382
I20260808 15:02:27.963734 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:27.964572 2771 external_mini_cluster.cc:1428] Setting key 504d993862a3d2a9a07b9dd735ec1e63
I20260808 15:02:27.969338 4492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de: Bootstrap starting.
I20260808 15:02:27.972733 4492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:27.973933 4492 log.cc:824] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:27.977116 4492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de: No bootstrap required, opened a new log
Aug 08 15:02:27 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201347, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:27.984754 4492 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9ff7104f524d64b169c939d9fb48de" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34003 } }
I20260808 15:02:27.985088 4492 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:27.985205 4492 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c9ff7104f524d64b169c939d9fb48de, State: Initialized, Role: FOLLOWER
I20260808 15:02:27.985803 4492 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9ff7104f524d64b169c939d9fb48de" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34003 } }
I20260808 15:02:27.986053 4492 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:27.986195 4492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:27.986362 4492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:27.986624 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:27.966517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34750 (local address 127.2.180.254:34003)
0808 15:02:27.967276 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:27.967290 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:27.967333 (+ 43us) server_negotiation.cc:407] Connection header received
0808 15:02:27.968972 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:27.968996 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:27.969338 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:27.969671 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:27.971325 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:27.972658 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:27.974322 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:27.975319 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:27.975434 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:27.979912 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:27.979921 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:27.979934 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:27.979973 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:27.982831 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:27.983450 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:27.983456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:27.983462 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:27.983633 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:27.984038 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:27.984041 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:27.984043 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:27.984507 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:27.984802 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:27.985213 (+ 411us) server_negotiation.cc:299] Negotiation successful
0808 15:02:27.985610 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":335,"threads_started":1}
I20260808 15:02:27.988663 4492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9ff7104f524d64b169c939d9fb48de" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34003 } }
I20260808 15:02:27.989202 4492 leader_election.cc:302] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c9ff7104f524d64b169c939d9fb48de; no voters:
I20260808 15:02:27.989714 4492 leader_election.cc:288] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:27.989816 4501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:27.990443 4501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [term 1 LEADER]: Becoming Leader. State: Replica: 0c9ff7104f524d64b169c939d9fb48de, State: Running, Role: LEADER
I20260808 15:02:27.991011 4501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9ff7104f524d64b169c939d9fb48de" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34003 } }
I20260808 15:02:27.991724 4492 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:27.993680 4503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c9ff7104f524d64b169c939d9fb48de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9ff7104f524d64b169c939d9fb48de" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34003 } } }
I20260808 15:02:27.994002 4503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:27.993654 4502 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c9ff7104f524d64b169c939d9fb48de. Latest consensus state: current_term: 1 leader_uuid: "0c9ff7104f524d64b169c939d9fb48de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c9ff7104f524d64b169c939d9fb48de" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34003 } } }
I20260808 15:02:27.994884 4502 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:27.997756 4506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:28.001561 4506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:28.009677 4506 catalog_manager.cc:1347] Generated new cluster ID: 72db6d72220a4929a01f27c8fc8f2ff7
I20260808 15:02:28.009765 4506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:28.020208 4506 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:28.021333 4506 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:28.027485 4506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de: Generated new TSK 0
I20260808 15:02:28.028177 4506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:02:28.076997 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:34003
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:28.385638 4531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:28.385914 4531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:28.386041 4531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:28.393095 4531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:28.393203 4531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:28.393328 4531 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:02:28.393437 4531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:28.402683 4531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:28.403864 4531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:28.405108 4531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:28.417649 4542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:28.417714 4540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:28.417752 4539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:28.418517 4531 server_base.cc:1034] running on GCE node
I20260808 15:02:28.419090 4531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:28.419929 4531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:28.421142 4531 hybrid_clock.cc:648] HybridClock initialized: now 1786201348421083 us; error 29 us; skew 500 ppm
I20260808 15:02:28.421398 4531 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:28 dist-test-slave-jlzn krb5kdc[4364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:28.424306 4531 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:02:28.425647 4531 webserver.cc:460] Webserver started at http://127.2.180.193:45409/ using document root <none> and password file <none>
I20260808 15:02:28.426257 4531 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:28.426352 4531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:28.426672 4531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:28.429441 4531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "8e81a7182e78400d9791d595f216fa7b"
format_stamp: "Formatted at 2026-08-08 15:02:28 on dist-test-slave-jlzn"
server_key: "1e2dfc8486bd4ccbfe5565ca990b5acf"
server_key_iv: "1a5d6c16597ca41b09b27149418093e5"
server_key_version: "encryptionkey@0"
I20260808 15:02:28.430199 4531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "8e81a7182e78400d9791d595f216fa7b"
format_stamp: "Formatted at 2026-08-08 15:02:28 on dist-test-slave-jlzn"
server_key: "1e2dfc8486bd4ccbfe5565ca990b5acf"
server_key_iv: "1a5d6c16597ca41b09b27149418093e5"
server_key_version: "encryptionkey@0"
I20260808 15:02:28.435151 4531 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 15:02:28.438516 4552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:28.439760 4531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:02:28.439944 4531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "8e81a7182e78400d9791d595f216fa7b"
format_stamp: "Formatted at 2026-08-08 15:02:28 on dist-test-slave-jlzn"
server_key: "1e2dfc8486bd4ccbfe5565ca990b5acf"
server_key_iv: "1a5d6c16597ca41b09b27149418093e5"
server_key_version: "encryptionkey@0"
I20260808 15:02:28.440147 4531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:28.457649 4531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:28.461418 4531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:28.461642 4531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:28.464231 4531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:28.464310 4531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:28.464426 4531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:28.464483 4531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:02:28 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:28.487371 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:28.464811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:33169 (local address 127.2.180.254:34003)
0808 15:02:28.464985 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:28.464991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:28.466184 (+ 1193us) server_negotiation.cc:407] Connection header received
0808 15:02:28.468889 (+ 2705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:28.468894 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:28.468969 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:28.469084 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:28.471366 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.472333 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.475163 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.475899 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.476021 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:28.480348 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:28.480356 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:28.480361 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:28.480407 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:28.483742 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:28.484770 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:28.484776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:28.484780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:28.484964 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:28.485563 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:28.485567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:28.485569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:28.485847 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:28.485971 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:28.486824 (+ 853us) server_negotiation.cc:299] Negotiation successful
0808 15:02:28.487041 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:02:28.488416 4565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:28.465171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:34003 (local address 127.2.180.193:33169)
0808 15:02:28.465985 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:28.466036 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:28.468572 (+ 2536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:28.469358 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:28.469370 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:28.470200 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:28.471156 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:28.471187 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.472524 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.472531 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:28.474947 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:28.474963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.476236 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.476244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:28.476501 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:28.477248 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:28.477273 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:28.480129 (+ 2856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:28.483914 (+ 3785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:28.483922 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:28.483940 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:28.484555 (+ 615us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:28.485088 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:28.485093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:28.485097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:28.485404 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:28.485983 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:28.485991 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:28.486408 (+ 417us) client_negotiation.cc:769] Sending connection context
0808 15:02:28.486743 (+ 335us) client_negotiation.cc:240] Negotiation successful
0808 15:02:28.487243 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":303,"threads_started":1}
I20260808 15:02:28.507332 4654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:28.493978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:34003 (local address 127.2.180.193:50865)
0808 15:02:28.494525 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:28.494553 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:28.494690 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:28.495532 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:28.495537 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:28.496186 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:28.496713 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:28.496724 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.498020 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.498028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:28.499424 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:28.499440 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.500382 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.500389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:28.500542 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:28.501064 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:28.501080 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:28.501774 (+ 694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:28.504623 (+ 2849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:28.504629 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:28.504633 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:28.505033 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:28.505382 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:28.505387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:28.505391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:28.505549 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:28.506145 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:28.506150 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:28.506370 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 15:02:28.506478 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 15:02:28.506808 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":346,"threads_started":1}
I20260808 15:02:28.507714 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:28.493804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:50865 (local address 127.2.180.254:34003)
0808 15:02:28.493946 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:28.493950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:28.494640 (+ 690us) server_negotiation.cc:407] Connection header received
0808 15:02:28.495355 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:28.495360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:28.495419 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:28.495496 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:28.496890 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.497820 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.499618 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.500191 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.500306 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:28.501953 (+ 1647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:28.501957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:28.501961 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:28.501988 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:28.504488 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:28.505144 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:28.505150 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:28.505154 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:28.505279 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:28.505706 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:28.505711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:28.505715 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:28.505998 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:28.507126 (+ 1128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:28.507245 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 15:02:28.507461 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 15:02:28.516287 4531 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:37105
I20260808 15:02:28.516330 4715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:37105 every 8 connection(s)
I20260808 15:02:28.517386 4531 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:28.519181 4716 heartbeater.cc:347] Connected to a master server at 127.2.180.254:34003
I20260808 15:02:28.519392 4716 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:28.520056 4716 heartbeater.cc:511] Master 127.2.180.254:34003 requested a full tablet report, sending...
I20260808 15:02:28.521822 4428 ts_manager.cc:194] Registered new tserver with Master: 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105)
I20260808 15:02:28.522493 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 4531
I20260808 15:02:28.522686 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:28.523279 2771 external_mini_cluster.cc:1428] Setting key 3407d6aeac9766e1d47f4fe0b32170e5
I20260808 15:02:28.523243 4428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:33169
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:02:28.560928 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:34003
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:28.864372 4722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:28.864634 4722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:28.864784 4722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:28.872601 4722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:28.872699 4722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:28.872819 4722 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:02:28.872916 4722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:28.882047 4722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:28.883143 4722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:28.884446 4722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:28.897639 4729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:28.897785 4733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:28.898206 4722 server_base.cc:1034] running on GCE node
W20260808 15:02:28.898257 4728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:28.898993 4722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:28.899825 4722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:28.901044 4722 hybrid_clock.cc:648] HybridClock initialized: now 1786201348900980 us; error 39 us; skew 500 ppm
I20260808 15:02:28.901314 4722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:28 dist-test-slave-jlzn krb5kdc[4364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:28.908221 4722 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:02:28.909534 4722 webserver.cc:460] Webserver started at http://127.2.180.194:46335/ using document root <none> and password file <none>
I20260808 15:02:28.910178 4722 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:28.910279 4722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:28.910580 4722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:28.913530 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "2f959366e61c46359e93a17cf4a3345d"
format_stamp: "Formatted at 2026-08-08 15:02:28 on dist-test-slave-jlzn"
server_key: "461eb7e78f9db754da003e3dbdab3823"
server_key_iv: "ce55a5f02a42cbcc3140b07fc6fd64b0"
server_key_version: "encryptionkey@0"
I20260808 15:02:28.914287 4722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "2f959366e61c46359e93a17cf4a3345d"
format_stamp: "Formatted at 2026-08-08 15:02:28 on dist-test-slave-jlzn"
server_key: "461eb7e78f9db754da003e3dbdab3823"
server_key_iv: "ce55a5f02a42cbcc3140b07fc6fd64b0"
server_key_version: "encryptionkey@0"
I20260808 15:02:28.918979 4722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:02:28.922180 4745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:28.923377 4722 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:02:28.923511 4722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "2f959366e61c46359e93a17cf4a3345d"
format_stamp: "Formatted at 2026-08-08 15:02:28 on dist-test-slave-jlzn"
server_key: "461eb7e78f9db754da003e3dbdab3823"
server_key_iv: "ce55a5f02a42cbcc3140b07fc6fd64b0"
server_key_version: "encryptionkey@0"
I20260808 15:02:28.923705 4722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:28.957839 4722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:28.962625 4722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:28.962903 4722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:28.965866 4722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:28.965972 4722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:28.966121 4722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:28.966202 4722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:02:28 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201348, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:28.984656 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:28.966011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:58335 (local address 127.2.180.254:34003)
0808 15:02:28.966211 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:28.966218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:28.967693 (+ 1475us) server_negotiation.cc:407] Connection header received
0808 15:02:28.969709 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:28.969715 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:28.969796 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:28.969883 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:28.971361 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.972201 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.974777 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.975447 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.975563 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:28.979292 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:28.979301 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:28.979305 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:28.979347 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:28.981765 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:28.982414 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:28.982419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:28.982421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:28.982546 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:28.982950 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:28.982954 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:28.982957 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:28.983217 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:28.983345 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:28.984011 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 15:02:28.984284 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 15:02:28.985297 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:28.966604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:34003 (local address 127.2.180.194:58335)
0808 15:02:28.967537 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:28.967586 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:28.969455 (+ 1869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:28.970152 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:28.970164 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:28.970683 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:28.971197 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:28.971218 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.972364 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.972370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:28.974589 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:28.974608 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.975735 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.975744 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:28.975976 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:28.976557 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:28.976578 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:28.979079 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:28.981892 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:28.981900 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:28.981913 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:28.982303 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:28.982647 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:28.982652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:28.982656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:28.982838 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:28.983340 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:28.983346 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:28.983643 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 15:02:28.983889 (+ 246us) client_negotiation.cc:240] Negotiation successful
0808 15:02:28.984233 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":396,"threads_started":1}
I20260808 15:02:29.006212 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:28.992447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:57959 (local address 127.2.180.254:34003)
0808 15:02:28.992636 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:28.992643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:28.993456 (+ 813us) server_negotiation.cc:407] Connection header received
0808 15:02:28.993575 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:28.993580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:28.993659 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:28.993755 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:28.994673 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.995849 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.997530 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.997933 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.998040 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.000175 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:29.000181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:29.000186 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:29.000223 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:29.003469 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.004140 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.004146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.004148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.004301 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.004693 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.004697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.004699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.005116 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:29.005276 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.005666 (+ 390us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.005875 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:02:29.006265 4823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:28.992753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:34003 (local address 127.2.180.194:57959)
0808 15:02:28.993355 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:28.993370 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:28.993466 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:28.993814 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:28.993818 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:28.994277 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:28.994548 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:28.994556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.996004 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.996009 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:28.997378 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:28.997391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:28.998119 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:28.998126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:28.998351 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:28.998925 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:28.998949 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:28.999971 (+ 1022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:29.003623 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:29.003626 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:29.003629 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:29.004036 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:29.004423 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:29.004431 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:29.004435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:29.004573 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:29.005242 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:29.005248 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:29.005419 (+ 171us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.005543 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.005837 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":321,"threads_started":1}
I20260808 15:02:29.024313 4722 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:44463
I20260808 15:02:29.024340 4913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:44463 every 8 connection(s)
I20260808 15:02:29.025409 4722 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:29.027326 4914 heartbeater.cc:347] Connected to a master server at 127.2.180.254:34003
I20260808 15:02:29.027534 4914 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:29.028232 4914 heartbeater.cc:511] Master 127.2.180.254:34003 requested a full tablet report, sending...
I20260808 15:02:29.029436 4428 ts_manager.cc:194] Registered new tserver with Master: 2f959366e61c46359e93a17cf4a3345d (127.2.180.194:44463)
I20260808 15:02:29.030217 4428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:58335
I20260808 15:02:29.034773 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 4722
I20260808 15:02:29.034947 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:29.035358 2771 external_mini_cluster.cc:1428] Setting key 6c349dcda5b79d7ef02a141797811209
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:02:29.072189 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:34003
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:29.379629 4918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:29.379882 4918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:29.380023 4918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:29.386974 4918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:29.387077 4918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:29.387203 4918 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:02:29.387329 4918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:29.396764 4918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:34003
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:29.397848 4918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:29.399078 4918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:29.411006 4924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:29.411100 4928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:29.411149 4925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:29.412133 4918 server_base.cc:1034] running on GCE node
I20260808 15:02:29.412730 4918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:29.413563 4918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:29.414772 4918 hybrid_clock.cc:648] HybridClock initialized: now 1786201349414704 us; error 39 us; skew 500 ppm
I20260808 15:02:29.415042 4918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:29 dist-test-slave-jlzn krb5kdc[4364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201349, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:29.418031 4918 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:02:29.419337 4918 webserver.cc:460] Webserver started at http://127.2.180.195:46863/ using document root <none> and password file <none>
I20260808 15:02:29.419997 4918 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:29.420107 4918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:29.420408 4918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:29.423262 4918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "7f0e842452304a328ae93945e75a403e"
format_stamp: "Formatted at 2026-08-08 15:02:29 on dist-test-slave-jlzn"
server_key: "b32293a65e452f00c4b10208486c9c1f"
server_key_iv: "a79ed1f81663eedae1714d42ed68ff28"
server_key_version: "encryptionkey@0"
I20260808 15:02:29.424100 4918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "7f0e842452304a328ae93945e75a403e"
format_stamp: "Formatted at 2026-08-08 15:02:29 on dist-test-slave-jlzn"
server_key: "b32293a65e452f00c4b10208486c9c1f"
server_key_iv: "a79ed1f81663eedae1714d42ed68ff28"
server_key_version: "encryptionkey@0"
I20260808 15:02:29.428982 4918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 15:02:29.432130 4943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:29.433279 4918 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:02:29.433429 4918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "7f0e842452304a328ae93945e75a403e"
format_stamp: "Formatted at 2026-08-08 15:02:29 on dist-test-slave-jlzn"
server_key: "b32293a65e452f00c4b10208486c9c1f"
server_key_iv: "a79ed1f81663eedae1714d42ed68ff28"
server_key_version: "encryptionkey@0"
I20260808 15:02:29.433627 4918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:29.462374 4918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:29.466053 4918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:29.466292 4918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:29.468873 4918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:29.468966 4918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:29.469115 4918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:29.469190 4918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:02:29 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201349, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:29.490581 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.469250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:59509 (local address 127.2.180.254:34003)
0808 15:02:29.469411 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.469415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.470693 (+ 1278us) server_negotiation.cc:407] Connection header received
0808 15:02:29.472711 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.472718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.472804 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.472890 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:29.474828 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.476027 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.478362 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.478980 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.479148 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.483760 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:29.483767 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:29.483771 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:29.483826 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:29.487283 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.488231 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.488238 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.488242 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.488420 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.488829 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.488834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.488838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.489197 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:29.489358 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.489972 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.490236 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:02:29.491822 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.469705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:34003 (local address 127.2.180.195:59509)
0808 15:02:29.470510 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:29.470568 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:29.472452 (+ 1884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:29.473117 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:29.473126 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:29.473798 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:29.474580 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.474609 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.476218 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.476228 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.478165 (+ 1937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.478198 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.479255 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.479263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.479619 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.480332 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:29.480366 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:29.483537 (+ 3171us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:29.487428 (+ 3891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:29.487438 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:29.487456 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:29.488104 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:29.488527 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:29.488530 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:29.488533 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:29.488729 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:29.489307 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:29.489312 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:29.489595 (+ 283us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.489855 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.490216 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":351,"threads_started":1}
I20260808 15:02:29.509606 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.498161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:43753 (local address 127.2.180.254:34003)
0808 15:02:29.498291 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.498295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.498931 (+ 636us) server_negotiation.cc:407] Connection header received
0808 15:02:29.499077 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.499080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.499135 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.499210 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:29.500111 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.500786 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.502012 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.502410 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.502513 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.504298 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:29.504303 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:29.504306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:29.504334 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:29.506940 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.507722 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.507727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.507731 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.507918 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.508241 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.508246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.508249 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.508494 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:29.508632 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.509040 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.509246 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:02:29.509742 5063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.498287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:34003 (local address 127.2.180.195:43753)
0808 15:02:29.498838 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:29.498854 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:29.498971 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:29.499248 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:29.499252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:29.499683 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:29.499987 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.499996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.500902 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.500907 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.501869 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.501880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.502582 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.502589 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.502728 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.503259 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:29.503275 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:29.504115 (+ 840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:29.507058 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:29.507063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:29.507067 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:29.507566 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:29.508014 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:29.508019 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:29.508023 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:29.508140 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:29.508607 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:29.508612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:29.508785 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.508920 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.509246 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":309,"threads_started":1}
I20260808 15:02:29.518887 5117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:37013 every 8 connection(s)
I20260808 15:02:29.518883 4918 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:37013
I20260808 15:02:29.520382 4918 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:29.522300 5118 heartbeater.cc:347] Connected to a master server at 127.2.180.254:34003
I20260808 15:02:29.522492 5118 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:29.523056 5118 heartbeater.cc:511] Master 127.2.180.254:34003 requested a full tablet report, sending...
I20260808 15:02:29.524255 4428 ts_manager.cc:194] Registered new tserver with Master: 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013)
I20260808 15:02:29.525027 4428 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:59509
I20260808 15:02:29.525849 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 4918
I20260808 15:02:29.525928 4716 heartbeater.cc:503] Master 127.2.180.254:34003 was elected leader, sending a full tablet report...
I20260808 15:02:29.526043 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:29.526585 2771 external_mini_cluster.cc:1428] Setting key 9908b98c746f052aee9b28226246b635
I20260808 15:02:29.529423 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 15:02:29 dist-test-slave-jlzn krb5kdc[4364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201349, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:02:29 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201349, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:29.573290 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.560486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34754 (local address 127.2.180.254:34003)
0808 15:02:29.560700 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.560706 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.560978 (+ 272us) server_negotiation.cc:407] Connection header received
0808 15:02:29.561156 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.561160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.561222 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.561303 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:29.562673 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.563389 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.564675 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.565113 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.565222 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.568677 (+ 3455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:29.568682 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:29.568685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:29.568721 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:29.570875 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.571460 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.571466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.571469 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.571609 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.572004 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.572007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.572009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.572270 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:29.572487 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.572812 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.573018 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:02:29.576893 4428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:02:29.583465 4428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:29.614432 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.595676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.2.180.195:37013)
0808 15:02:29.596415 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.596424 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.607791 (+ 11367us) server_negotiation.cc:407] Connection header received
0808 15:02:29.607963 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.607972 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.608211 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.608378 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:29.608980 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.610030 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.612274 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.613328 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.613437 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.613664 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.613965 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.614113 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":383,"threads_started":1}
I20260808 15:02:29.614786 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.595016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:37013 (local address 127.0.0.1:50818)
0808 15:02:29.607681 (+ 12665us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:29.607695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:29.607825 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:29.608405 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:29.608413 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:29.608440 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:29.608810 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.608820 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.610154 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.610160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.612130 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.612147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.613551 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.613557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.613757 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.613794 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.613897 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.614083 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12567,"thread_start_us":12889,"threads_started":1}
I20260808 15:02:29.618290 5012 tablet_service.cc:1467] Processing CreateTablet for tablet 95e75728ccad47ad8a691090f63c440d (DEFAULT_TABLE table=test-table [id=8268db78a23c4ea48b5723dd268a439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:29.619937 5012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e75728ccad47ad8a691090f63c440d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.626016 5139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.597739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:44463 (local address 127.0.0.1:55116)
0808 15:02:29.615880 (+ 18141us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:29.615899 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:29.615990 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:29.618302 (+ 2312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:29.618306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:29.618332 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:29.618504 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.618514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.620256 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.620262 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.622472 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.622489 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.625179 (+ 2690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.625185 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.625381 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.625418 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.625483 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.625619 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18032,"mutex_wait_us":186,"thread_start_us":7302,"threads_started":1}
I20260808 15:02:29.627087 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.598300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55116 (local address 127.2.180.194:44463)
0808 15:02:29.617695 (+ 19395us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.617705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.617736 (+ 31us) server_negotiation.cc:407] Connection header received
0808 15:02:29.617852 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.617860 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.618089 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.618294 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:29.618700 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.620073 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.622646 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.624982 (+ 2336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.626359 (+ 1377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.626554 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.626617 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.626785 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19206,"thread_start_us":20139,"threads_started":1}
I20260808 15:02:29.630055 5145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.594817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37105 (local address 127.0.0.1:56562)
0808 15:02:29.605468 (+ 10651us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:29.605514 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:29.605691 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:29.619790 (+ 14099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:29.619797 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:29.619844 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:29.620341 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.620355 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.624747 (+ 4392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.624753 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.627068 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.627085 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.629312 (+ 2227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.629318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.629498 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.629529 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.629591 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.629719 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10383,"mutex_wait_us":3,"thread_start_us":33417,"threads_started":1}
I20260808 15:02:29.630172 4818 tablet_service.cc:1467] Processing CreateTablet for tablet 95e75728ccad47ad8a691090f63c440d (DEFAULT_TABLE table=test-table [id=8268db78a23c4ea48b5723dd268a439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:29.630970 5138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.595049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56562 (local address 127.2.180.193:37105)
0808 15:02:29.619032 (+ 23983us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.619041 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.619070 (+ 29us) server_negotiation.cc:407] Connection header received
0808 15:02:29.619166 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.619174 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.619386 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.622715 (+ 3329us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:29.622935 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.624564 (+ 1629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.627343 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.629090 (+ 1747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.630279 (+ 1189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.630467 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.630529 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.630692 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23807,"thread_start_us":24467,"threads_started":1}
I20260808 15:02:29.631367 4818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e75728ccad47ad8a691090f63c440d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.633783 5150 tablet_bootstrap.cc:492] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Bootstrap starting.
I20260808 15:02:29.634224 4620 tablet_service.cc:1467] Processing CreateTablet for tablet 95e75728ccad47ad8a691090f63c440d (DEFAULT_TABLE table=test-table [id=8268db78a23c4ea48b5723dd268a439f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:29.635524 4620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95e75728ccad47ad8a691090f63c440d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.637470 5150 tablet_bootstrap.cc:654] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.638701 5150 log.cc:824] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:29.642714 5153 tablet_bootstrap.cc:492] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Bootstrap starting.
I20260808 15:02:29.646175 5153 tablet_bootstrap.cc:654] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.647306 5153 log.cc:824] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:29.648146 5154 tablet_bootstrap.cc:492] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Bootstrap starting.
I20260808 15:02:29.648968 5150 tablet_bootstrap.cc:492] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: No bootstrap required, opened a new log
I20260808 15:02:29.649217 5150 ts_tablet_manager.cc:1397] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260808 15:02:29.651077 5154 tablet_bootstrap.cc:654] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.651283 5153 tablet_bootstrap.cc:492] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: No bootstrap required, opened a new log
I20260808 15:02:29.651549 5153 ts_tablet_manager.cc:1397] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 15:02:29.652140 5154 log.cc:824] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:29.654346 5154 tablet_bootstrap.cc:492] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: No bootstrap required, opened a new log
I20260808 15:02:29.654565 5154 ts_tablet_manager.cc:1397] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260808 15:02:29.659291 5150 raft_consensus.cc:348] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.659708 5150 raft_consensus.cc:374] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.659832 5150 raft_consensus.cc:729] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0e842452304a328ae93945e75a403e, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.660423 5150 consensus_queue.cc:260] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.661528 5118 heartbeater.cc:503] Master 127.2.180.254:34003 was elected leader, sending a full tablet report...
I20260808 15:02:29.661484 5153 raft_consensus.cc:348] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.661868 5153 raft_consensus.cc:374] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.661970 5153 raft_consensus.cc:729] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f959366e61c46359e93a17cf4a3345d, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.662179 5150 ts_tablet_manager.cc:1428] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 15:02:29.662602 5153 consensus_queue.cc:260] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.662669 5154 raft_consensus.cc:348] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.663054 5154 raft_consensus.cc:374] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.663153 5154 raft_consensus.cc:729] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e81a7182e78400d9791d595f216fa7b, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.663712 5154 consensus_queue.cc:260] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.664867 5153 ts_tablet_manager.cc:1428] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Time spent starting tablet: real 0.013s user 0.008s sys 0.006s
I20260808 15:02:29.665207 4914 heartbeater.cc:503] Master 127.2.180.254:34003 was elected leader, sending a full tablet report...
I20260808 15:02:29.667369 5154 ts_tablet_manager.cc:1428] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260808 15:02:29.692271 5159 raft_consensus.cc:482] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:29.692495 5159 raft_consensus.cc:504] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.694097 5159 leader_election.cc:288] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105), 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013)
I20260808 15:02:29.700742 5138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.694602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:34085 (local address 127.2.180.193:37105)
0808 15:02:29.694808 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.694814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.695388 (+ 574us) server_negotiation.cc:407] Connection header received
0808 15:02:29.695587 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.695593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.695650 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.695725 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:29.696305 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.697041 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.698848 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.699774 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.699941 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.700014 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.700389 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.700521 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 15:02:29.700973 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.694312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37105 (local address 127.2.180.194:34085)
0808 15:02:29.695282 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:29.695306 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:29.695457 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:29.695822 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:29.695827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:29.695853 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:29.696159 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.696169 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.697152 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.697157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.698699 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.698712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.700035 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.700042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.700227 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.700262 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.700350 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.700468 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":35,"thread_start_us":691,"threads_started":1}
I20260808 15:02:29.701490 5143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.694706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:33007 (local address 127.2.180.195:37013)
0808 15:02:29.694868 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.694874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.695724 (+ 850us) server_negotiation.cc:407] Connection header received
0808 15:02:29.695895 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.695899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.695956 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.696029 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:29.696608 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.697364 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.699191 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.700388 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.700523 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.700572 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.701183 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.701315 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:02:29.701629 5165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.694478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:37013 (local address 127.2.180.194:33007)
0808 15:02:29.695631 (+ 1153us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:29.695646 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:29.695752 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:29.696063 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:29.696067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:29.696091 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:29.696469 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.696479 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.697497 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.697501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.699007 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.699020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.700705 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.700710 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.700991 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.701015 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.701147 (+ 132us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.701263 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1045,"mutex_wait_us":16,"thread_start_us":857,"threads_started":1}
I20260808 15:02:29.701974 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e75728ccad47ad8a691090f63c440d" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e81a7182e78400d9791d595f216fa7b" is_pre_election: true
I20260808 15:02:29.702395 4653 raft_consensus.cc:2454] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 0.
I20260808 15:02:29.702473 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e75728ccad47ad8a691090f63c440d" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0e842452304a328ae93945e75a403e" is_pre_election: true
I20260808 15:02:29.702943 5040 raft_consensus.cc:2454] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 0.
I20260808 15:02:29.702976 4749 leader_election.cc:302] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f959366e61c46359e93a17cf4a3345d, 8e81a7182e78400d9791d595f216fa7b; no voters:
I20260808 15:02:29.703343 5159 raft_consensus.cc:2781] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:29.703506 5159 raft_consensus.cc:482] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:29.703613 5159 raft_consensus.cc:3037] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:29.705730 5159 raft_consensus.cc:504] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.706239 5159 leader_election.cc:288] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 election: Requested vote from peers 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105), 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013)
I20260808 15:02:29.706580 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e75728ccad47ad8a691090f63c440d" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e81a7182e78400d9791d595f216fa7b"
I20260808 15:02:29.706642 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95e75728ccad47ad8a691090f63c440d" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0e842452304a328ae93945e75a403e"
I20260808 15:02:29.706748 4653 raft_consensus.cc:3037] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:29.706821 5040 raft_consensus.cc:3037] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:29.708868 4653 raft_consensus.cc:2454] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 1.
I20260808 15:02:29.709040 5040 raft_consensus.cc:2454] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 1.
I20260808 15:02:29.709218 4749 leader_election.cc:302] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f959366e61c46359e93a17cf4a3345d, 8e81a7182e78400d9791d595f216fa7b; no voters:
I20260808 15:02:29.709479 5159 raft_consensus.cc:2781] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:29.709879 5159 raft_consensus.cc:686] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 1 LEADER]: Becoming Leader. State: Replica: 2f959366e61c46359e93a17cf4a3345d, State: Running, Role: LEADER
I20260808 15:02:29.710289 5159 consensus_queue.cc:237] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } }
I20260808 15:02:29.714279 4427 catalog_manager.cc:5495] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f959366e61c46359e93a17cf4a3345d (127.2.180.194). New cstate: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } health_report { overall_health: HEALTHY } } }
I20260808 15:02:29.761404 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.756104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55126 (local address 127.2.180.194:44463)
0808 15:02:29.756308 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.756314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.756432 (+ 118us) server_negotiation.cc:407] Connection header received
0808 15:02:29.756726 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.756731 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.756808 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.756924 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:29.757286 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.757984 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.759283 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.759744 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.759888 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.760352 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:29.760819 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:29.760938 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.761073 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.761163 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:02:29.768882 4653 raft_consensus.cc:1261] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Refusing update from remote peer 2f959366e61c46359e93a17cf4a3345d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:29.768879 5040 raft_consensus.cc:1261] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Refusing update from remote peer 2f959366e61c46359e93a17cf4a3345d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260808 15:02:29.769426 4717 tablet.cc:2369] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:02:29.769757 5169 consensus_queue.cc:1035] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:29.770099 5159 consensus_queue.cc:1035] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:02:29.771908 5119 tablet.cc:2369] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:02:29.777462 4915 tablet.cc:2369] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:02:29.781443 5175 mvcc.cc:205] Tried to move back new op lower bound from 7316280728644030464 to 7316280728416632832. Current Snapshot: MvccSnapshot[applied={T|T < 7316280728644030464}]
I20260808 15:02:29.796739 4428 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:29.796978 4428 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:29.800484 4428 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de: Sending DeleteTablet for 3 replicas of tablet 95e75728ccad47ad8a691090f63c440d
I20260808 15:02:29.801578 4620 tablet_service.cc:1514] Processing DeleteTablet for tablet 95e75728ccad47ad8a691090f63c440d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:29 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:56562
I20260808 15:02:29.801584 5012 tablet_service.cc:1514] Processing DeleteTablet for tablet 95e75728ccad47ad8a691090f63c440d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:29 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:50818
I20260808 15:02:29.802280 4818 tablet_service.cc:1514] Processing DeleteTablet for tablet 95e75728ccad47ad8a691090f63c440d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:29 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:55116
I20260808 15:02:29.802502 5186 tablet_replica.cc:330] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: stopping tablet replica
I20260808 15:02:29.802917 5186 raft_consensus.cc:2229] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:29.803424 5186 raft_consensus.cc:2258] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:29.804672 5191 tablet_replica.cc:330] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: stopping tablet replica
I20260808 15:02:29.804492 4428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 15:02:29.805009 5191 raft_consensus.cc:2229] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 1 LEADER]: Raft consensus shutting down.
W20260808 15:02:29.805111 4428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:29.805585 5191 raft_consensus.cc:2258] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:29.807454 5191 ts_tablet_manager.cc:1905] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:29.808571 5186 ts_tablet_manager.cc:1905] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:29.812654 5189 tablet_replica.cc:330] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: stopping tablet replica
I20260808 15:02:29.813033 5189 raft_consensus.cc:2229] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:29.813577 5189 raft_consensus.cc:2258] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:29.814935 5012 tablet_service.cc:1467] Processing CreateTablet for tablet cc66100e3d5446c09cc4feb425a0ef4a (DEFAULT_TABLE table=test-table [id=d5e0862ed2104f9c8fee74ded19ff959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:29.815163 4818 tablet_service.cc:1467] Processing CreateTablet for tablet cc66100e3d5446c09cc4feb425a0ef4a (DEFAULT_TABLE table=test-table [id=d5e0862ed2104f9c8fee74ded19ff959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:29.815346 5012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc66100e3d5446c09cc4feb425a0ef4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.815261 4620 tablet_service.cc:1467] Processing CreateTablet for tablet cc66100e3d5446c09cc4feb425a0ef4a (DEFAULT_TABLE table=test-table [id=d5e0862ed2104f9c8fee74ded19ff959]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:29.815590 4818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc66100e3d5446c09cc4feb425a0ef4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.820297 5150 tablet_bootstrap.cc:492] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: Bootstrap starting.
I20260808 15:02:29.820873 5191 ts_tablet_manager.cc:1918] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:02:29.821055 5191 log.cc:1196] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/95e75728ccad47ad8a691090f63c440d
I20260808 15:02:29.821614 5191 ts_tablet_manager.cc:1939] T 95e75728ccad47ad8a691090f63c440d P 2f959366e61c46359e93a17cf4a3345d: Deleting consensus metadata
I20260808 15:02:29.821883 5153 tablet_bootstrap.cc:492] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: Bootstrap starting.
I20260808 15:02:29.821964 5150 tablet_bootstrap.cc:654] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.823480 5153 tablet_bootstrap.cc:654] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.823457 5189 ts_tablet_manager.cc:1905] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:29.824024 4620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc66100e3d5446c09cc4feb425a0ef4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.830178 4404 catalog_manager.cc:4841] TS 2f959366e61c46359e93a17cf4a3345d (127.2.180.194:44463): tablet 95e75728ccad47ad8a691090f63c440d (table test-table [id=8268db78a23c4ea48b5723dd268a439f]) successfully deleted
I20260808 15:02:29.830515 5154 tablet_bootstrap.cc:492] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: Bootstrap starting.
I20260808 15:02:29.832093 5154 tablet_bootstrap.cc:654] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.837630 5150 tablet_bootstrap.cc:492] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: No bootstrap required, opened a new log
I20260808 15:02:29.837613 5153 tablet_bootstrap.cc:492] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: No bootstrap required, opened a new log
I20260808 15:02:29.837776 5153 ts_tablet_manager.cc:1397] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.013s
I20260808 15:02:29.837815 5150 ts_tablet_manager.cc:1397] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.004s
I20260808 15:02:29.838636 5153 raft_consensus.cc:348] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.838891 5153 raft_consensus.cc:374] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.838997 5153 raft_consensus.cc:729] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f959366e61c46359e93a17cf4a3345d, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.839301 5153 consensus_queue.cc:260] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.840018 5153 ts_tablet_manager.cc:1428] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 15:02:29.840479 5150 raft_consensus.cc:348] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.840777 5150 raft_consensus.cc:374] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.840941 5150 raft_consensus.cc:729] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0e842452304a328ae93945e75a403e, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.841375 5150 consensus_queue.cc:260] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.844388 5150 ts_tablet_manager.cc:1428] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260808 15:02:29.845755 5186 ts_tablet_manager.cc:1918] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:02:29.845928 5186 log.cc:1196] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/95e75728ccad47ad8a691090f63c440d
I20260808 15:02:29.846457 5186 ts_tablet_manager.cc:1939] T 95e75728ccad47ad8a691090f63c440d P 8e81a7182e78400d9791d595f216fa7b: Deleting consensus metadata
I20260808 15:02:29.848454 4408 catalog_manager.cc:4841] TS 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105): tablet 95e75728ccad47ad8a691090f63c440d (table test-table [id=8268db78a23c4ea48b5723dd268a439f]) successfully deleted
I20260808 15:02:29.849119 5154 tablet_bootstrap.cc:492] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: No bootstrap required, opened a new log
I20260808 15:02:29.849272 5154 ts_tablet_manager.cc:1397] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.004s
I20260808 15:02:29.850009 5154 raft_consensus.cc:348] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.850237 5154 raft_consensus.cc:374] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.850337 5154 raft_consensus.cc:729] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e81a7182e78400d9791d595f216fa7b, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.850461 5189 ts_tablet_manager.cc:1918] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:02:29.850608 5189 log.cc:1196] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/95e75728ccad47ad8a691090f63c440d
I20260808 15:02:29.850586 5154 consensus_queue.cc:260] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.851125 5189 ts_tablet_manager.cc:1939] T 95e75728ccad47ad8a691090f63c440d P 7f0e842452304a328ae93945e75a403e: Deleting consensus metadata
I20260808 15:02:29.851146 5154 ts_tablet_manager.cc:1428] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 15:02:29.852794 4406 catalog_manager.cc:4841] TS 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013): tablet 95e75728ccad47ad8a691090f63c440d (table test-table [id=8268db78a23c4ea48b5723dd268a439f]) successfully deleted
I20260808 15:02:29.867738 5169 raft_consensus.cc:482] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:29.867993 5169 raft_consensus.cc:504] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.868707 5169 leader_election.cc:288] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105), 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013)
I20260808 15:02:29.869148 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc66100e3d5446c09cc4feb425a0ef4a" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0e842452304a328ae93945e75a403e" is_pre_election: true
I20260808 15:02:29.869148 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc66100e3d5446c09cc4feb425a0ef4a" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e81a7182e78400d9791d595f216fa7b" is_pre_election: true
I20260808 15:02:29.869347 5040 raft_consensus.cc:2454] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 0.
I20260808 15:02:29.869362 4653 raft_consensus.cc:2454] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 0.
I20260808 15:02:29.869719 4747 leader_election.cc:302] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f959366e61c46359e93a17cf4a3345d, 7f0e842452304a328ae93945e75a403e; no voters:
I20260808 15:02:29.870049 5169 raft_consensus.cc:2781] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:29.870157 5169 raft_consensus.cc:482] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:29.870271 5169 raft_consensus.cc:3037] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:29.872570 5169 raft_consensus.cc:504] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.873140 5169 leader_election.cc:288] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 election: Requested vote from peers 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105), 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013)
I20260808 15:02:29.873507 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc66100e3d5446c09cc4feb425a0ef4a" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e81a7182e78400d9791d595f216fa7b"
I20260808 15:02:29.873557 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc66100e3d5446c09cc4feb425a0ef4a" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0e842452304a328ae93945e75a403e"
I20260808 15:02:29.873677 4653 raft_consensus.cc:3037] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:29.873749 5040 raft_consensus.cc:3037] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:29.875520 4653 raft_consensus.cc:2454] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 1.
I20260808 15:02:29.875919 4749 leader_election.cc:302] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f959366e61c46359e93a17cf4a3345d, 8e81a7182e78400d9791d595f216fa7b; no voters:
I20260808 15:02:29.876159 5169 raft_consensus.cc:2781] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:29.876319 5169 raft_consensus.cc:686] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 1 LEADER]: Becoming Leader. State: Replica: 2f959366e61c46359e93a17cf4a3345d, State: Running, Role: LEADER
I20260808 15:02:29.876659 5040 raft_consensus.cc:2454] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 1.
I20260808 15:02:29.876572 5169 consensus_queue.cc:237] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.878961 4428 catalog_manager.cc:5495] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f959366e61c46359e93a17cf4a3345d (127.2.180.194). New cstate: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } health_report { overall_health: UNKNOWN } } }
Aug 08 15:02:29 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201347, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:29.923789 4496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.909881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34770 (local address 127.2.180.254:34003)
0808 15:02:29.910056 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:29.910065 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:29.910609 (+ 544us) server_negotiation.cc:407] Connection header received
0808 15:02:29.910754 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:29.910760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:29.910837 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:29.910939 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:29.912076 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.913258 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.914532 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.915193 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.915321 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.918683 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:29.918687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:29.918691 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:29.918719 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:29.921117 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.921941 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.921945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.921947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.922070 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:29.922474 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:29.922477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:29.922479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:29.922723 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:29.922825 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:29.923309 (+ 484us) server_negotiation.cc:299] Negotiation successful
0808 15:02:29.923512 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":54}
I20260808 15:02:29.924202 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:29.909932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:34003 (local address 127.0.0.1:34770)
0808 15:02:29.910509 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:29.910526 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:29.910638 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:29.910953 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:29.910956 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:29.911602 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:29.911916 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.911927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.913381 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.913385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.914397 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:29.914409 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:29.915354 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:29.915359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:29.915485 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:29.916170 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:29.916187 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:29.918504 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:29.921250 (+ 2746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:29.921263 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:29.921271 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:29.921837 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:29.922167 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:29.922172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:29.922176 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:29.922375 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:29.922840 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:29.922848 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:29.923042 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 15:02:29.923240 (+ 198us) client_negotiation.cc:240] Negotiation successful
0808 15:02:29.923676 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":128,"thread_start_us":317,"threads_started":1}
I20260808 15:02:29.929219 4428 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:34770:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 15:02:29.930162 4428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:29.936439 4620 tablet_service.cc:1467] Processing CreateTablet for tablet 63c4f8388d994239861c4d1b2c179beb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=613dd3b49c9a4b6096a707b3a0d67f26]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:02:29.936534 4818 tablet_service.cc:1467] Processing CreateTablet for tablet 63c4f8388d994239861c4d1b2c179beb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=613dd3b49c9a4b6096a707b3a0d67f26]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:02:29.937045 4620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63c4f8388d994239861c4d1b2c179beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.937062 4818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63c4f8388d994239861c4d1b2c179beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.942453 5153 tablet_bootstrap.cc:492] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: Bootstrap starting.
I20260808 15:02:29.943789 5012 tablet_service.cc:1467] Processing CreateTablet for tablet 63c4f8388d994239861c4d1b2c179beb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=613dd3b49c9a4b6096a707b3a0d67f26]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:02:29.944272 5153 tablet_bootstrap.cc:654] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.944353 5012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 63c4f8388d994239861c4d1b2c179beb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:29.944952 5154 tablet_bootstrap.cc:492] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: Bootstrap starting.
I20260808 15:02:29.946583 5154 tablet_bootstrap.cc:654] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.951943 5150 tablet_bootstrap.cc:492] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: Bootstrap starting.
I20260808 15:02:29.953238 5150 tablet_bootstrap.cc:654] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:29.962639 5150 tablet_bootstrap.cc:492] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: No bootstrap required, opened a new log
I20260808 15:02:29.962855 5150 ts_tablet_manager.cc:1397] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260808 15:02:29.963598 5150 raft_consensus.cc:348] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.963925 5150 raft_consensus.cc:374] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.964064 5150 raft_consensus.cc:729] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f0e842452304a328ae93945e75a403e, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.964406 5150 consensus_queue.cc:260] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.965044 5158 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:29.965401 5158 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:29.968693 5153 tablet_bootstrap.cc:492] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: No bootstrap required, opened a new log
I20260808 15:02:29.968901 5153 ts_tablet_manager.cc:1397] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: Time spent bootstrapping tablet: real 0.026s user 0.010s sys 0.000s
I20260808 15:02:29.969657 5153 raft_consensus.cc:348] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.969957 5153 raft_consensus.cc:374] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.970072 5153 raft_consensus.cc:729] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f959366e61c46359e93a17cf4a3345d, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.970391 5153 consensus_queue.cc:260] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.971844 5150 ts_tablet_manager.cc:1428] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: Time spent starting tablet: real 0.009s user 0.001s sys 0.001s
I20260808 15:02:29.972334 5153 ts_tablet_manager.cc:1428] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260808 15:02:29.972216 5169 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:29.974629 5169 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:29.976816 5154 tablet_bootstrap.cc:492] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: No bootstrap required, opened a new log
I20260808 15:02:29.976989 5154 ts_tablet_manager.cc:1397] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.017s
I20260808 15:02:29.977695 5154 raft_consensus.cc:348] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.977941 5154 raft_consensus.cc:374] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:29.978032 5154 raft_consensus.cc:729] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e81a7182e78400d9791d595f216fa7b, State: Initialized, Role: FOLLOWER
I20260808 15:02:29.978271 5154 consensus_queue.cc:260] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:29.978888 5154 ts_tablet_manager.cc:1428] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 15:02:29.978754 5161 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:29.979105 5161 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:30.185386 5169 raft_consensus.cc:482] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:30.185587 5169 raft_consensus.cc:504] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:30.186249 5169 leader_election.cc:288] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105), 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013)
I20260808 15:02:30.186779 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63c4f8388d994239861c4d1b2c179beb" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e81a7182e78400d9791d595f216fa7b" is_pre_election: true
I20260808 15:02:30.186792 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63c4f8388d994239861c4d1b2c179beb" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0e842452304a328ae93945e75a403e" is_pre_election: true
I20260808 15:02:30.187043 4653 raft_consensus.cc:2454] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 0.
I20260808 15:02:30.187072 5040 raft_consensus.cc:2454] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 0.
I20260808 15:02:30.187498 4747 leader_election.cc:302] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f959366e61c46359e93a17cf4a3345d, 7f0e842452304a328ae93945e75a403e; no voters:
I20260808 15:02:30.187834 5169 raft_consensus.cc:2781] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:30.187960 5169 raft_consensus.cc:482] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:30.188071 5169 raft_consensus.cc:3037] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:30.190570 5169 raft_consensus.cc:504] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:30.191143 5169 leader_election.cc:288] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 election: Requested vote from peers 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105), 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013)
I20260808 15:02:30.191586 4653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63c4f8388d994239861c4d1b2c179beb" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e81a7182e78400d9791d595f216fa7b"
I20260808 15:02:30.191669 5040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "63c4f8388d994239861c4d1b2c179beb" candidate_uuid: "2f959366e61c46359e93a17cf4a3345d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f0e842452304a328ae93945e75a403e"
I20260808 15:02:30.191864 4653 raft_consensus.cc:3037] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:30.191900 5040 raft_consensus.cc:3037] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:30.194092 4653 raft_consensus.cc:2454] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 1.
I20260808 15:02:30.194448 4749 leader_election.cc:302] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f959366e61c46359e93a17cf4a3345d, 8e81a7182e78400d9791d595f216fa7b; no voters:
I20260808 15:02:30.194770 5169 raft_consensus.cc:2781] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:30.194815 5040 raft_consensus.cc:2454] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f959366e61c46359e93a17cf4a3345d in term 1.
I20260808 15:02:30.194913 5169 raft_consensus.cc:686] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [term 1 LEADER]: Becoming Leader. State: Replica: 2f959366e61c46359e93a17cf4a3345d, State: Running, Role: LEADER
I20260808 15:02:30.195195 5169 consensus_queue.cc:237] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } }
I20260808 15:02:30.197031 5159 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: TxnStatusTablet state changed. Reason: New leader 2f959366e61c46359e93a17cf4a3345d. Latest consensus state: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:30.197300 5159 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:02:30.198558 4428 catalog_manager.cc:5495] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f959366e61c46359e93a17cf4a3345d (127.2.180.194). New cstate: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } health_report { overall_health: UNKNOWN } } }
I20260808 15:02:30.200531 5212 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 15:02:30.200865 5212 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 15:02:30.266429 5204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.261142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:44463 (local address 127.0.0.1:55138)
0808 15:02:30.261350 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:30.261372 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:30.261511 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:30.261942 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:30.261946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:30.261956 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:02:30.262112 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:30.262120 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.263516 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.263519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:30.264518 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:30.264531 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.265458 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.265462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:30.265585 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.265637 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:02:30.266033 (+ 396us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:02:30.266043 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 15:02:30.266095 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 15:02:30.266205 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 15:02:30.266507 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.261307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55138 (local address 127.2.180.194:44463)
0808 15:02:30.261497 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:30.261503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:30.261524 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:02:30.261713 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:30.261720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:30.261811 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:30.261920 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:30.262247 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.263399 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.264657 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.265331 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.265469 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.265798 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:30.265949 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:30.266040 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:30.266126 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 15:02:30.266208 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 15:02:30.270510 5040 raft_consensus.cc:1261] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Refusing update from remote peer 2f959366e61c46359e93a17cf4a3345d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:30.270496 4653 raft_consensus.cc:1261] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Refusing update from remote peer 2f959366e61c46359e93a17cf4a3345d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:30.271107 5169 consensus_queue.cc:1035] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:30.271374 5159 consensus_queue.cc:1035] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:30.278724 5161 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: TxnStatusTablet state changed. Reason: New leader 2f959366e61c46359e93a17cf4a3345d. Latest consensus state: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:30.278985 5161 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:30.280385 5169 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:30.280666 5169 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:02:30.280956 5173 mvcc.cc:205] Tried to move back new op lower bound from 7316280730701729792 to 7316280730402140160. Current Snapshot: MvccSnapshot[applied={T|T < 7316280730701729792}]
I20260808 15:02:30.281886 5161 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:30.282114 5161 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 8e81a7182e78400d9791d595f216fa7b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:30.283478 5158 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: TxnStatusTablet state changed. Reason: New leader 2f959366e61c46359e93a17cf4a3345d. Latest consensus state: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:30.283735 5158 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:30.284405 5159 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:30.284657 5159 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 2f959366e61c46359e93a17cf4a3345d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:02:30.288112 5158 tablet_replica.cc:439] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2f959366e61c46359e93a17cf4a3345d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 } } peers { permanent_uuid: "2f959366e61c46359e93a17cf4a3345d" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 44463 } } peers { permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 } } }
I20260808 15:02:30.288341 5158 tablet_replica.cc:442] T 63c4f8388d994239861c4d1b2c179beb P 7f0e842452304a328ae93945e75a403e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:30.305119 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.299100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:53273 (local address 127.2.180.194:44463)
0808 15:02:30.299295 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:30.299301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:30.299846 (+ 545us) server_negotiation.cc:407] Connection header received
0808 15:02:30.300004 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:30.300010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:30.300107 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:30.300204 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:30.300639 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.301775 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.303195 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.303901 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.304020 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.304386 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:30.304523 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:30.304614 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:30.304710 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 15:02:30.304795 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51}
I20260808 15:02:30.305121 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.299135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:44463 (local address 127.2.180.194:53273)
0808 15:02:30.299741 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:30.299756 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:30.299900 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:30.300221 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:30.300224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:30.300233 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:02:30.300510 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:30.300518 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.301881 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.301884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:30.303065 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:30.303078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.304045 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.304049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:30.304179 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.304227 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:02:30.304611 (+ 384us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:02:30.304618 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 15:02:30.304676 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 15:02:30.304769 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":320,"threads_started":1}
I20260808 15:02:30.313931 5040 raft_consensus.cc:1261] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Refusing update from remote peer 2f959366e61c46359e93a17cf4a3345d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:30.314002 4653 raft_consensus.cc:1261] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Refusing update from remote peer 2f959366e61c46359e93a17cf4a3345d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:30.314548 5169 consensus_queue.cc:1035] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e81a7182e78400d9791d595f216fa7b" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:30.314795 5159 consensus_queue.cc:1035] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f0e842452304a328ae93945e75a403e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 37013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:30.380265 4426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:30.380467 4426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:30.383129 4426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c9ff7104f524d64b169c939d9fb48de: Sending DeleteTablet for 3 replicas of tablet cc66100e3d5446c09cc4feb425a0ef4a
I20260808 15:02:30.383876 4620 tablet_service.cc:1514] Processing DeleteTablet for tablet cc66100e3d5446c09cc4feb425a0ef4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:30 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:56562
I20260808 15:02:30.384058 4817 tablet_service.cc:1514] Processing DeleteTablet for tablet cc66100e3d5446c09cc4feb425a0ef4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:30 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:55116
I20260808 15:02:30.384300 5012 tablet_service.cc:1514] Processing DeleteTablet for tablet cc66100e3d5446c09cc4feb425a0ef4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:30 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:50818
I20260808 15:02:30.399364 5232 tablet_replica.cc:330] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: stopping tablet replica
I20260808 15:02:30.399663 5230 tablet_replica.cc:330] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: stopping tablet replica
I20260808 15:02:30.399699 5232 raft_consensus.cc:2229] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:30.399870 5230 raft_consensus.cc:2229] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:30.400123 5232 raft_consensus.cc:2258] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:30.400123 5230 raft_consensus.cc:2258] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:30.401199 5230 ts_tablet_manager.cc:1905] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:30.401533 5232 ts_tablet_manager.cc:1905] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:30.404822 5231 tablet_replica.cc:330] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: stopping tablet replica
I20260808 15:02:30.405022 5231 raft_consensus.cc:2229] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:02:30.405433 5231 raft_consensus.cc:2258] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:30.407868 5230 ts_tablet_manager.cc:1918] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:30.408063 5230 log.cc:1196] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/cc66100e3d5446c09cc4feb425a0ef4a
I20260808 15:02:30.408558 5230 ts_tablet_manager.cc:1939] T cc66100e3d5446c09cc4feb425a0ef4a P 8e81a7182e78400d9791d595f216fa7b: Deleting consensus metadata
I20260808 15:02:30.409631 4408 catalog_manager.cc:4841] TS 8e81a7182e78400d9791d595f216fa7b (127.2.180.193:37105): tablet cc66100e3d5446c09cc4feb425a0ef4a (table test-table [id=d5e0862ed2104f9c8fee74ded19ff959]) successfully deleted
I20260808 15:02:30.409790 5231 ts_tablet_manager.cc:1905] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:30.411532 5232 ts_tablet_manager.cc:1918] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:30.411749 5232 log.cc:1196] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/cc66100e3d5446c09cc4feb425a0ef4a
I20260808 15:02:30.412343 5232 ts_tablet_manager.cc:1939] T cc66100e3d5446c09cc4feb425a0ef4a P 7f0e842452304a328ae93945e75a403e: Deleting consensus metadata
I20260808 15:02:30.413725 4406 catalog_manager.cc:4841] TS 7f0e842452304a328ae93945e75a403e (127.2.180.195:37013): tablet cc66100e3d5446c09cc4feb425a0ef4a (table test-table [id=d5e0862ed2104f9c8fee74ded19ff959]) successfully deleted
Aug 08 15:02:30 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201349, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:30.417531 5231 ts_tablet_manager.cc:1918] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:30.417663 5231 log.cc:1196] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/cc66100e3d5446c09cc4feb425a0ef4a
I20260808 15:02:30.418138 5231 ts_tablet_manager.cc:1939] T cc66100e3d5446c09cc4feb425a0ef4a P 2f959366e61c46359e93a17cf4a3345d: Deleting consensus metadata
I20260808 15:02:30.419191 4404 catalog_manager.cc:4841] TS 2f959366e61c46359e93a17cf4a3345d (127.2.180.194:44463): tablet cc66100e3d5446c09cc4feb425a0ef4a (table test-table [id=d5e0862ed2104f9c8fee74ded19ff959]) successfully deleted
I20260808 15:02:30.421794 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.395993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56570 (local address 127.2.180.193:37105)
0808 15:02:30.397057 (+ 1064us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:30.397062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:30.400422 (+ 3360us) server_negotiation.cc:407] Connection header received
0808 15:02:30.401256 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:30.401262 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:30.401361 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:30.401441 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:30.405757 (+ 4316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.407055 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.410060 (+ 3005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.410724 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.410819 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.416031 (+ 5212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:30.416051 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:30.416066 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:30.416103 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:30.418949 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.419695 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.419705 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.419709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.419923 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.420275 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.420280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.420284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.420703 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:30.421018 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:30.421185 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 15:02:30.421417 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":926,"thread_start_us":812,"threads_started":1}
W20260808 15:02:30.422586 4710 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56570
I20260808 15:02:30.438632 5241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.425519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34772 (local address 127.2.180.254:34003)
0808 15:02:30.426052 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:30.426056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:30.426071 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:02:30.426116 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:30.426119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:30.426186 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:30.426268 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:30.427816 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.428662 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.430398 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.430777 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.430858 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.433534 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:30.433539 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:30.433543 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:30.433572 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:30.436095 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.436818 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.436822 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.436824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.436968 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.437347 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.437350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.437352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.437645 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:30.437877 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:30.438169 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 15:02:30.438351 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":308,"threads_started":1}
W20260808 15:02:30.439348 4426 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34772
I20260808 15:02:30.441167 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4531
I20260808 15:02:30.453958 5234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.441964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.2.180.193:37105)
0808 15:02:30.442148 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:30.442153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:30.442546 (+ 393us) server_negotiation.cc:407] Connection header received
0808 15:02:30.442643 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:30.442648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:30.442734 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:30.442853 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:30.443961 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.444973 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.446078 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.446704 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.446818 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.448712 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:30.448718 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:30.448722 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:30.448759 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:30.451752 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.452332 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.452337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.452341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.452480 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.452756 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.452760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.452763 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.453051 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:30.453288 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:30.453453 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 15:02:30.453653 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 15:02:30.454519 4710 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56582
W20260808 15:02:30.454977 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4531: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:30.455127 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 4531
I20260808 15:02:30.478333 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4722
Aug 08 15:02:30 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201349, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:02:30.494371 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.478997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55150 (local address 127.2.180.194:44463)
0808 15:02:30.479231 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:30.479238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:30.479259 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:02:30.479320 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:30.479325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:30.479400 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:30.479514 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:30.480588 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.481698 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.482856 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.483598 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.483775 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.487395 (+ 3620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:30.487412 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:30.487433 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:30.487480 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:30.491470 (+ 3990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.492137 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.492146 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.492150 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.492340 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.492678 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.492683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.492690 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.493181 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:30.493486 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:30.493677 (+ 191us) server_negotiation.cc:299] Negotiation successful
0808 15:02:30.493946 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 15:02:30.495225 4907 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55150
W20260808 15:02:30.495882 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4722: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:30.496034 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 4722
I20260808 15:02:30.525285 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4918
Aug 08 15:02:30 dist-test-slave-jlzn krb5kdc[4364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201349, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:02:30.542657 5245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:30.526013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.2.180.195:37013)
0808 15:02:30.526535 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:30.526539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:30.526560 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:02:30.526631 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:30.526635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:30.526696 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:30.526780 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:30.528359 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.529135 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.530862 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:30.531282 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:30.531370 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:30.536246 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:30.536260 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:30.536279 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:30.536320 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:30.540055 (+ 3735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.540652 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.540662 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.540665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.540860 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:30.541174 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:30.541179 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:30.541183 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:30.541585 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:30.541871 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:30.542023 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 15:02:30.542252 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":260,"threads_started":1}
W20260808 15:02:30.543385 5112 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50832
W20260808 15:02:30.544011 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4918: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:30.544148 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 4918
I20260808 15:02:30.566069 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4382
I20260808 15:02:30.567047 4485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:30.677433 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 4382
2026-08-08T15:02:30Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4454 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:30 dist-test-slave-jlzn krb5kdc[5249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:30 dist-test-slave-jlzn krb5kdc[5249](info): set up 2 sockets
Aug 08 15:02:30 dist-test-slave-jlzn krb5kdc[5249](info): commencing operation
krb5kdc: starting...
W20260808 15:02:31.848016 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:31 dist-test-slave-jlzn krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:32.016983 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45149
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:32.321193 5269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:32.321448 5269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:32.321583 5269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:32.328534 5269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:32.328621 5269 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 15:02:32.328683 5269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:32.328768 5269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:32.328832 5269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:32.328914 5269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:32.338120 5269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45149
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:32.339218 5269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:32.340504 5269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:32.351454 5274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:32.351491 5279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:32.351558 5276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:32.352219 5269 server_base.cc:1034] running on GCE node
I20260808 15:02:32.352902 5269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:32.354126 5269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:32.355347 5269 hybrid_clock.cc:648] HybridClock initialized: now 1786201352355292 us; error 25 us; skew 500 ppm
I20260808 15:02:32.355677 5269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:32 dist-test-slave-jlzn krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:32.358594 5269 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:32.359900 5269 webserver.cc:460] Webserver started at http://127.2.180.254:34433/ using document root <none> and password file <none>
I20260808 15:02:32.360512 5269 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:32.360621 5269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:32.360910 5269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:32.363636 5269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3"
format_stamp: "Formatted at 2026-08-08 15:02:32 on dist-test-slave-jlzn"
server_key: "8d73dbe54b87b395aa773b99f2b021ea"
server_key_iv: "55af45c823995dcd15ab5e92af3fbb25"
server_key_version: "encryptionkey@0"
I20260808 15:02:32.364391 5269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3"
format_stamp: "Formatted at 2026-08-08 15:02:32 on dist-test-slave-jlzn"
server_key: "8d73dbe54b87b395aa773b99f2b021ea"
server_key_iv: "55af45c823995dcd15ab5e92af3fbb25"
server_key_version: "encryptionkey@0"
I20260808 15:02:32.369071 5269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:02:32.372241 5289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:32.373394 5269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:02:32.373548 5269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3"
format_stamp: "Formatted at 2026-08-08 15:02:32 on dist-test-slave-jlzn"
server_key: "8d73dbe54b87b395aa773b99f2b021ea"
server_key_iv: "55af45c823995dcd15ab5e92af3fbb25"
server_key_version: "encryptionkey@0"
I20260808 15:02:32.373731 5269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:32.398049 5269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:32.401614 5269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:32.401845 5269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:32.421648 5269 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:45149
I20260808 15:02:32.421716 5382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:45149 every 8 connection(s)
I20260808 15:02:32.422938 5269 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:32.426554 5384 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:32.430138 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 5269
I20260808 15:02:32.430357 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:32.431136 2771 external_mini_cluster.cc:1428] Setting key a759f1cf61ad99bf805d11b3d89a0bc0
I20260808 15:02:32.438709 5384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: Bootstrap starting.
I20260808 15:02:32.441550 5384 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:32.442552 5384 log.cc:824] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:02:32 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:32.445075 5384 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: No bootstrap required, opened a new log
I20260808 15:02:32.451633 5387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:32.433194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.2.180.254:45149)
0808 15:02:32.433941 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:32.433953 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:32.433993 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:02:32.435547 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:32.435569 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:32.435972 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:32.436327 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:32.437767 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:32.439066 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:32.440330 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:32.440959 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:32.441136 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:32.444895 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:32.444903 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:32.444917 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:32.444955 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:32.447954 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:32.448576 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:32.448584 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:32.448589 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:32.448755 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:32.449104 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:32.449109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:32.449112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:32.449611 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:32.449901 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:32.450330 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 15:02:32.450700 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":350,"threads_started":1}
I20260808 15:02:32.452617 5384 raft_consensus.cc:348] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45149 } }
I20260808 15:02:32.452957 5384 raft_consensus.cc:374] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:32.453052 5384 raft_consensus.cc:729] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b01dc10bb6894d1a8ce75c3eef3b10c3, State: Initialized, Role: FOLLOWER
I20260808 15:02:32.453565 5384 consensus_queue.cc:260] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45149 } }
I20260808 15:02:32.453794 5384 raft_consensus.cc:388] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:32.453925 5384 raft_consensus.cc:482] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:32.454084 5384 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:32.456173 5384 raft_consensus.cc:504] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45149 } }
I20260808 15:02:32.456657 5384 leader_election.cc:302] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b01dc10bb6894d1a8ce75c3eef3b10c3; no voters:
I20260808 15:02:32.457273 5384 leader_election.cc:288] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:32.457376 5389 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:32.458009 5389 raft_consensus.cc:686] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [term 1 LEADER]: Becoming Leader. State: Replica: b01dc10bb6894d1a8ce75c3eef3b10c3, State: Running, Role: LEADER
I20260808 15:02:32.458564 5389 consensus_queue.cc:237] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45149 } }
I20260808 15:02:32.459285 5384 sys_catalog.cc:564] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:32.461227 5391 sys_catalog.cc:455] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45149 } } }
I20260808 15:02:32.461575 5391 sys_catalog.cc:458] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:32.463894 5401 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:32.463754 5392 sys_catalog.cc:455] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b01dc10bb6894d1a8ce75c3eef3b10c3. Latest consensus state: current_term: 1 leader_uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b01dc10bb6894d1a8ce75c3eef3b10c3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45149 } } }
I20260808 15:02:32.464089 5392 sys_catalog.cc:458] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:32.468176 5401 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:32.475889 5401 catalog_manager.cc:1347] Generated new cluster ID: 47a39cc2803e4fd78f492766f1f5f90a
I20260808 15:02:32.475993 5401 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:32.496662 5401 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:32.497859 5401 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:32.506506 5401 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: Generated new TSK 0
I20260808 15:02:32.507239 5401 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:02:32.559338 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45149
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:32.874925 5418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:32.875209 5418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:32.875360 5418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:32.882462 5418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:32.882581 5418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:32.882700 5418 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 15:02:32.882767 5418 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:02:32.882869 5418 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:32.893620 5418 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:32.896171 5418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:32.897689 5418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:32.910422 5426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:32.910451 5429 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:32.910493 5431 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:32.911211 5418 server_base.cc:1034] running on GCE node
I20260808 15:02:32.911837 5418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:32.912714 5418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:32.913923 5418 hybrid_clock.cc:648] HybridClock initialized: now 1786201352913873 us; error 33 us; skew 500 ppm
I20260808 15:02:32.914223 5418 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:32 dist-test-slave-jlzn krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:32.917240 5418 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:02:32.918556 5418 webserver.cc:460] Webserver started at http://127.2.180.193:43271/ using document root <none> and password file <none>
I20260808 15:02:32.919169 5418 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:32.919282 5418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:32.919574 5418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:32.922317 5418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "6ba7224d134745559ef0e91280c2496d"
format_stamp: "Formatted at 2026-08-08 15:02:32 on dist-test-slave-jlzn"
server_key: "a72f1edf538a9677db8962a1af0fc9fa"
server_key_iv: "9dc99fd67fadf5b764fbd8944656897b"
server_key_version: "encryptionkey@0"
I20260808 15:02:32.923056 5418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "6ba7224d134745559ef0e91280c2496d"
format_stamp: "Formatted at 2026-08-08 15:02:32 on dist-test-slave-jlzn"
server_key: "a72f1edf538a9677db8962a1af0fc9fa"
server_key_iv: "9dc99fd67fadf5b764fbd8944656897b"
server_key_version: "encryptionkey@0"
I20260808 15:02:32.927882 5418 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 15:02:32.931079 5444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:32.932341 5418 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:02:32.932492 5418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "6ba7224d134745559ef0e91280c2496d"
format_stamp: "Formatted at 2026-08-08 15:02:32 on dist-test-slave-jlzn"
server_key: "a72f1edf538a9677db8962a1af0fc9fa"
server_key_iv: "9dc99fd67fadf5b764fbd8944656897b"
server_key_version: "encryptionkey@0"
I20260808 15:02:32.932659 5418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:32.960861 5418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:32.964601 5418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:32.964844 5418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:32.967342 5418 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:32.967446 5418 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:32.967554 5418 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:32.967628 5418 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:02:32 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:32.989257 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:32.967860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:59903 (local address 127.2.180.254:45149)
0808 15:02:32.970217 (+ 2357us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:32.970221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:32.970240 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:02:32.972346 (+ 2106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:32.972352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:32.972438 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:32.972580 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:32.974474 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:32.975152 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:32.978119 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:32.978653 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:32.978779 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:32.982971 (+ 4192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:32.982975 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:32.982978 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:32.983015 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:32.985993 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:32.986914 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:32.986920 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:32.986924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:32.987044 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:32.987464 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:32.987470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:32.987473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:32.987862 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:32.987998 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:32.988661 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 15:02:32.988877 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2227,"thread_start_us":2141,"threads_started":1}
I20260808 15:02:32.990100 5463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:32.968507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45149 (local address 127.2.180.193:59903)
0808 15:02:32.969452 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:32.969509 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:32.972024 (+ 2515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:32.972827 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:32.972839 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:32.973540 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:32.974303 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:32.974327 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:32.975290 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:32.975297 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:32.977939 (+ 2642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:32.977959 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:32.978925 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:32.978938 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:32.979225 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:32.979972 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:32.980006 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:32.982745 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:32.986151 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:32.986160 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:32.986173 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:32.986792 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:32.987149 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:32.987154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:32.987158 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:32.987344 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:32.988011 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:32.988018 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:32.988316 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 15:02:32.988540 (+ 224us) client_negotiation.cc:240] Negotiation successful
0808 15:02:32.989013 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":384,"threads_started":1}
I20260808 15:02:33.008765 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:32.995504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:43273 (local address 127.2.180.254:45149)
0808 15:02:32.995681 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:32.995686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:32.996257 (+ 571us) server_negotiation.cc:407] Connection header received
0808 15:02:32.996448 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:32.996454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:32.996516 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:32.996585 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:32.997742 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:32.998413 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.000328 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.000823 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.000942 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.003051 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:33.003056 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:33.003059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:33.003087 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:33.005808 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.006529 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.006534 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.006538 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.006713 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.007291 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.007296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.007299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.007588 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:33.007725 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:33.008235 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 15:02:33.008450 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:02:33.008810 5533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:32.995458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45149 (local address 127.2.180.193:43273)
0808 15:02:32.996140 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:32.996164 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:32.996319 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:32.996645 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:32.996650 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:32.997204 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:32.997603 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:32.997615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:32.998515 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:32.998519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.000173 (+ 1654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.000195 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.001049 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.001057 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.001266 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.001876 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:33.001904 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:33.002880 (+ 976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:33.005964 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.005972 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.005976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.006417 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.006883 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.006889 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.006893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.007152 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.007708 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:33.007713 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:33.007934 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 15:02:33.008074 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 15:02:33.008368 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":358,"threads_started":1}
I20260808 15:02:33.021201 5418 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:34459
I20260808 15:02:33.021243 5616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:34459 every 8 connection(s)
I20260808 15:02:33.022344 5418 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:33.024147 5617 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45149
I20260808 15:02:33.024353 5617 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:33.024938 5617 heartbeater.cc:511] Master 127.2.180.254:45149 requested a full tablet report, sending...
I20260808 15:02:33.026719 5313 ts_manager.cc:194] Registered new tserver with Master: 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459)
I20260808 15:02:33.028123 5313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:59903
I20260808 15:02:33.028213 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 5418
I20260808 15:02:33.028388 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:33.029067 2771 external_mini_cluster.cc:1428] Setting key 8d0534f579a0bc5df1a3488b8525e3d0
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:02:33.066816 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45149
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:33.372745 5621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:33.372993 5621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:33.373113 5621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:33.380102 5621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:33.380203 5621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:33.380318 5621 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 15:02:33.380378 5621 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:02:33.380478 5621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:33.389696 5621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:33.390858 5621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:33.392167 5621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:33.404452 5628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:33.404495 5630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:33.404531 5632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:33.405340 5621 server_base.cc:1034] running on GCE node
I20260808 15:02:33.405903 5621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:33.406750 5621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:33.407934 5621 hybrid_clock.cc:648] HybridClock initialized: now 1786201353407886 us; error 42 us; skew 500 ppm
I20260808 15:02:33.408234 5621 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:33 dist-test-slave-jlzn krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:33.411099 5621 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:02:33.412528 5621 webserver.cc:460] Webserver started at http://127.2.180.194:39053/ using document root <none> and password file <none>
I20260808 15:02:33.413137 5621 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:33.413262 5621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:33.413553 5621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:33.416385 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "5f7069b5af8642afae5dadc19dac2185"
format_stamp: "Formatted at 2026-08-08 15:02:33 on dist-test-slave-jlzn"
server_key: "9d31d303512eb7335942ce29d9aa64bc"
server_key_iv: "e331f911d868ff40cbc2b04dcb917bd4"
server_key_version: "encryptionkey@0"
I20260808 15:02:33.417095 5621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "5f7069b5af8642afae5dadc19dac2185"
format_stamp: "Formatted at 2026-08-08 15:02:33 on dist-test-slave-jlzn"
server_key: "9d31d303512eb7335942ce29d9aa64bc"
server_key_iv: "e331f911d868ff40cbc2b04dcb917bd4"
server_key_version: "encryptionkey@0"
I20260808 15:02:33.422163 5621 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 15:02:33.426218 5646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:33.427727 5621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:02:33.427906 5621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "5f7069b5af8642afae5dadc19dac2185"
format_stamp: "Formatted at 2026-08-08 15:02:33 on dist-test-slave-jlzn"
server_key: "9d31d303512eb7335942ce29d9aa64bc"
server_key_iv: "e331f911d868ff40cbc2b04dcb917bd4"
server_key_version: "encryptionkey@0"
I20260808 15:02:33.428108 5621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:33.445304 5621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:33.448905 5621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:33.449133 5621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:33.451568 5621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:33.451663 5621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:33.451828 5621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:33.451973 5621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:02:33 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:33.472764 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:33.452013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:52893 (local address 127.2.180.254:45149)
0808 15:02:33.452209 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:33.452216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:33.453436 (+ 1220us) server_negotiation.cc:407] Connection header received
0808 15:02:33.455407 (+ 1971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:33.455413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:33.455497 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:33.455623 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:33.457653 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.458611 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.461410 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.461985 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.462131 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.466450 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:33.466454 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:33.466457 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:33.466489 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:33.469108 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.470059 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.470065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.470068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.470223 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.470744 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.470749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.470752 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.471137 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:33.471283 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:33.472180 (+ 897us) server_negotiation.cc:299] Negotiation successful
0808 15:02:33.472403 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:02:33.473670 5659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:33.452449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45149 (local address 127.2.180.194:52893)
0808 15:02:33.453275 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:33.453318 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:33.455124 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:33.455883 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:33.455899 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:33.456612 (+ 713us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:33.457427 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.457461 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.458792 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.458797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.461270 (+ 2473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.461285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.462293 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.462301 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.462596 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.463375 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:33.463408 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:33.466241 (+ 2833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:33.469273 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.469279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.469289 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.469875 (+ 586us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.470322 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.470327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.470330 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.470602 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.471307 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:33.471315 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:33.471694 (+ 379us) client_negotiation.cc:769] Sending connection context
0808 15:02:33.472062 (+ 368us) client_negotiation.cc:240] Negotiation successful
0808 15:02:33.472555 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":343,"threads_started":1}
I20260808 15:02:33.496865 5621 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:39841
I20260808 15:02:33.496912 5808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:39841 every 8 connection(s)
I20260808 15:02:33.498497 5621 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:33.498513 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:33.483560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:34905 (local address 127.2.180.254:45149)
0808 15:02:33.483710 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:33.483714 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:33.484535 (+ 821us) server_negotiation.cc:407] Connection header received
0808 15:02:33.484740 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:33.484747 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:33.484840 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:33.484932 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:33.486270 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.487397 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.489215 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.489835 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.489963 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.492343 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:33.492349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:33.492354 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:33.492390 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:33.495537 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.496478 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.496483 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.496486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.496659 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.497093 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.497099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.497103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.497402 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:33.497512 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:33.498053 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 15:02:33.498303 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 15:02:33.498670 5753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:33.483652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45149 (local address 127.2.180.194:34905)
0808 15:02:33.484409 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:33.484429 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:33.484589 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:33.485001 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:33.485006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:33.485617 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:33.486102 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.486117 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.487556 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.487564 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.489063 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.489076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.490033 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.490039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.490164 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.490813 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:33.490839 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:33.492125 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:33.495664 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.495670 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.495678 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.496332 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.496789 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.496794 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.496797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.496971 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.497519 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:33.497524 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:33.497743 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 15:02:33.497925 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 15:02:33.498251 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":435,"threads_started":1}
I20260808 15:02:33.500761 5810 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45149
I20260808 15:02:33.500998 5810 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:33.501791 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 5621
I20260808 15:02:33.501806 5810 heartbeater.cc:511] Master 127.2.180.254:45149 requested a full tablet report, sending...
I20260808 15:02:33.501967 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:33.502524 2771 external_mini_cluster.cc:1428] Setting key b71bf9297b049d197368e403f3804e96
I20260808 15:02:33.503295 5313 ts_manager.cc:194] Registered new tserver with Master: 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841)
I20260808 15:02:33.504434 5313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:52893
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:02:33.540411 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45149
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:45903
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:33.849323 5822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:33.849581 5822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:33.849679 5822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:33.856573 5822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:33.856667 5822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:33.856752 5822 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 15:02:33.856804 5822 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:02:33.856869 5822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:33.866163 5822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:45903
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:45149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:33.867244 5822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:33.868461 5822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:33.880882 5829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:33.880841 5830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:33.880904 5832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:33.881796 5822 server_base.cc:1034] running on GCE node
I20260808 15:02:33.882323 5822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:33.883208 5822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:33.884397 5822 hybrid_clock.cc:648] HybridClock initialized: now 1786201353884338 us; error 47 us; skew 500 ppm
I20260808 15:02:33.884694 5822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:33 dist-test-slave-jlzn krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:33.887698 5822 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:02:33.889081 5822 webserver.cc:460] Webserver started at http://127.2.180.195:45429/ using document root <none> and password file <none>
I20260808 15:02:33.889693 5822 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:33.889808 5822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:33.890103 5822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:33.892876 5822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "bc667d285fd7413dab838a30cfaf9103"
format_stamp: "Formatted at 2026-08-08 15:02:33 on dist-test-slave-jlzn"
server_key: "18bf6e93c5afae425b3062eb9eda21ab"
server_key_iv: "6c0c12c6b999ab08920d6f3cfbf7b0a2"
server_key_version: "encryptionkey@0"
I20260808 15:02:33.893635 5822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "bc667d285fd7413dab838a30cfaf9103"
format_stamp: "Formatted at 2026-08-08 15:02:33 on dist-test-slave-jlzn"
server_key: "18bf6e93c5afae425b3062eb9eda21ab"
server_key_iv: "6c0c12c6b999ab08920d6f3cfbf7b0a2"
server_key_version: "encryptionkey@0"
I20260808 15:02:33.898502 5822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 15:02:33.901758 5844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:33.902956 5822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:02:33.903107 5822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "bc667d285fd7413dab838a30cfaf9103"
format_stamp: "Formatted at 2026-08-08 15:02:33 on dist-test-slave-jlzn"
server_key: "18bf6e93c5afae425b3062eb9eda21ab"
server_key_iv: "6c0c12c6b999ab08920d6f3cfbf7b0a2"
server_key_version: "encryptionkey@0"
I20260808 15:02:33.903275 5822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:33.923213 5822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:33.926847 5822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:33.927093 5822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:33.929589 5822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:33.929709 5822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:33.929812 5822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:33.929919 5822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:02:33 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201353, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:33.950883 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:33.929946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:34977 (local address 127.2.180.254:45149)
0808 15:02:33.930178 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:33.930185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:33.931440 (+ 1255us) server_negotiation.cc:407] Connection header received
0808 15:02:33.933538 (+ 2098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:33.933546 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:33.933637 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:33.933742 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:33.935784 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.936926 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.939299 (+ 2373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.939779 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.939931 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.944406 (+ 4475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:33.944413 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:33.944419 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:33.944464 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:33.947686 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.948367 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.948373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.948377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.948552 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.949028 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.949034 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.949037 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.949408 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:33.949553 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:33.950243 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 15:02:33.950486 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 15:02:33.951439 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:33.930413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45149 (local address 127.2.180.195:34977)
0808 15:02:33.931258 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:33.931311 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:33.933272 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:33.934012 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:33.934020 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:33.934690 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:33.935564 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.935594 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.937084 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.937089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.939160 (+ 2071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.939175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.940100 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.940110 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.940416 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.941135 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:33.941160 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:33.944166 (+ 3006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:33.947849 (+ 3683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.947857 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.947875 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.948270 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.948683 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.948688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.948694 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.948902 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.949568 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:33.949576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:33.949871 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 15:02:33.950090 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 15:02:33.950417 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":346,"threads_started":1}
I20260808 15:02:33.969789 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:33.957560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:57781 (local address 127.2.180.254:45149)
0808 15:02:33.957723 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:33.957728 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:33.958462 (+ 734us) server_negotiation.cc:407] Connection header received
0808 15:02:33.958568 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:33.958575 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:33.958649 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:33.958746 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:33.959939 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.960806 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.962069 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.962449 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.962565 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.964167 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:33.964173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:33.964178 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:33.964207 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:33.967197 (+ 2990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.967831 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.967837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.967841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.967963 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:33.968429 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:33.968434 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:33.968438 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:33.968785 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:33.968932 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:33.969395 (+ 463us) server_negotiation.cc:299] Negotiation successful
0808 15:02:33.969560 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:02:33.970124 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:33.957650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45149 (local address 127.2.180.195:57781)
0808 15:02:33.958363 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:33.958384 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:33.958482 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:33.958800 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:33.958808 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:33.959455 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:33.959792 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.959823 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.960945 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.960951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.961936 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:33.961948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:33.962623 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:33.962629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:33.962786 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:33.963281 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:33.963297 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:33.963986 (+ 689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:33.967326 (+ 3340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.967332 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.967336 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.967703 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.968078 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:33.968083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:33.968091 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:33.968256 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:33.968943 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:33.968949 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:33.969165 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 15:02:33.969324 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 15:02:33.969643 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":420,"threads_started":1}
I20260808 15:02:33.980005 5822 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:32821
I20260808 15:02:33.980026 6015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:32821 every 8 connection(s)
I20260808 15:02:33.981135 5822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:33.982895 6017 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45149
I20260808 15:02:33.983111 6017 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:33.983690 6017 heartbeater.cc:511] Master 127.2.180.254:45149 requested a full tablet report, sending...
I20260808 15:02:33.984907 5313 ts_manager.cc:194] Registered new tserver with Master: bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821)
I20260808 15:02:33.985138 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 5822
I20260808 15:02:33.985291 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:33.985709 2771 external_mini_cluster.cc:1428] Setting key 329544b9ef858468711a48c1b4f00b81
I20260808 15:02:33.985788 5313 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:34977
I20260808 15:02:33.988399 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 15:02:34 dist-test-slave-jlzn krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:02:34 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:34.030668 5617 heartbeater.cc:503] Master 127.2.180.254:45149 was elected leader, sending a full tablet report...
I20260808 15:02:34.034575 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.020467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42280 (local address 127.2.180.254:45149)
0808 15:02:34.020681 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.020687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.021138 (+ 451us) server_negotiation.cc:407] Connection header received
0808 15:02:34.021327 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.021331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.021399 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.021485 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:34.023330 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.024136 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.025348 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.025790 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.025899 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.029389 (+ 3490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:34.029393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:34.029396 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:34.029429 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:34.032326 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:34.032932 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:34.032936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:34.032938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:34.033063 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:34.033355 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:34.033359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:34.033363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:34.033670 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:34.033852 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.034070 (+ 218us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.034227 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 15:02:34.038196 5313 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:02:34.044808 5313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:34.076800 6041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.056634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37786 (local address 127.2.180.194:39841)
0808 15:02:34.067671 (+ 11037us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.067679 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.068378 (+ 699us) server_negotiation.cc:407] Connection header received
0808 15:02:34.068690 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.068700 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.068923 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.069048 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:34.069768 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.071168 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.073959 (+ 2791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.075355 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.075514 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.075717 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.076344 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.076509 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10865,"thread_start_us":11044,"threads_started":1}
I20260808 15:02:34.076854 6042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.056269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:39841 (local address 127.0.0.1:37786)
0808 15:02:34.068267 (+ 11998us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:34.068285 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:34.068490 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:34.069283 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:34.069289 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:34.069311 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:34.069598 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.069607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.071323 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.071326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.073796 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.073810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.075873 (+ 2063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.075880 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.076076 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.076124 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 15:02:34.076250 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 15:02:34.076459 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11878,"thread_start_us":524,"threads_started":1}
I20260808 15:02:34.077751 6035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.058762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:32821 (local address 127.0.0.1:33254)
0808 15:02:34.070122 (+ 11360us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:34.070139 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:34.070252 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:34.070861 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:34.070866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:34.070890 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:34.071308 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.071320 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.073380 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.073386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.075134 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.075151 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.076954 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.076962 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.077126 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.077162 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:02:34.077240 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 15:02:34.077352 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11233,"mutex_wait_us":23,"thread_start_us":9799,"threads_started":1}
I20260808 15:02:34.078123 6043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.059470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.2.180.195:32821)
0808 15:02:34.066400 (+ 6930us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.066410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.070362 (+ 3952us) server_negotiation.cc:407] Connection header received
0808 15:02:34.070457 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.070464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.070672 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.070841 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:34.071535 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.073214 (+ 1679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.075303 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.076656 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.076773 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.077596 (+ 823us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.077662 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.077806 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6728,"thread_start_us":6518,"threads_started":1}
I20260808 15:02:34.078264 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.056269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.2.180.193:34459)
0808 15:02:34.070447 (+ 14178us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.070452 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.070474 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:02:34.070549 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.070554 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.070712 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.070881 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:34.071788 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.072915 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.075363 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.076767 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.076894 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.077085 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.077844 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.078001 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14045,"thread_start_us":13917,"threads_started":1}
I20260808 15:02:34.078662 6039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.056083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34459 (local address 127.0.0.1:39376)
0808 15:02:34.056921 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:34.056958 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:34.057109 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:34.071126 (+ 14017us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:34.071134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:34.071161 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:34.071625 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.071635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.073085 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.073089 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.075173 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.075187 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.077241 (+ 2054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.077248 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.077402 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.077430 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 15:02:34.077519 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 15:02:34.077656 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"mutex_wait_us":1,"thread_start_us":13768,"threads_started":1}
I20260808 15:02:34.080856 5713 tablet_service.cc:1467] Processing CreateTablet for tablet afc9253164ac46b68cc02d9b5f26cda3 (DEFAULT_TABLE table=test-table [id=9b1d2a8d7c544f5e9fe2422693474b7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:34.081578 5912 tablet_service.cc:1467] Processing CreateTablet for tablet afc9253164ac46b68cc02d9b5f26cda3 (DEFAULT_TABLE table=test-table [id=9b1d2a8d7c544f5e9fe2422693474b7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:34.082471 5713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc9253164ac46b68cc02d9b5f26cda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.082741 5912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc9253164ac46b68cc02d9b5f26cda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.082983 5516 tablet_service.cc:1467] Processing CreateTablet for tablet afc9253164ac46b68cc02d9b5f26cda3 (DEFAULT_TABLE table=test-table [id=9b1d2a8d7c544f5e9fe2422693474b7c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:34.084329 5516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afc9253164ac46b68cc02d9b5f26cda3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.093603 6044 tablet_bootstrap.cc:492] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Bootstrap starting.
I20260808 15:02:34.096225 6045 tablet_bootstrap.cc:492] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Bootstrap starting.
I20260808 15:02:34.097466 6047 tablet_bootstrap.cc:492] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Bootstrap starting.
I20260808 15:02:34.098134 6044 tablet_bootstrap.cc:654] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.099329 6045 tablet_bootstrap.cc:654] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.099395 6044 log.cc:824] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:34.100370 6047 tablet_bootstrap.cc:654] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.100454 6045 log.cc:824] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:34.101509 6047 log.cc:824] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:34.102157 6044 tablet_bootstrap.cc:492] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: No bootstrap required, opened a new log
I20260808 15:02:34.102432 6044 ts_tablet_manager.cc:1397] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.002s
I20260808 15:02:34.103227 6045 tablet_bootstrap.cc:492] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: No bootstrap required, opened a new log
I20260808 15:02:34.103497 6045 ts_tablet_manager.cc:1397] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260808 15:02:34.104470 6047 tablet_bootstrap.cc:492] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: No bootstrap required, opened a new log
I20260808 15:02:34.104727 6047 ts_tablet_manager.cc:1397] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:02:34.109941 6044 raft_consensus.cc:348] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.110280 6044 raft_consensus.cc:374] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.110375 6044 raft_consensus.cc:729] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc667d285fd7413dab838a30cfaf9103, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.110946 6044 consensus_queue.cc:260] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.111972 6017 heartbeater.cc:503] Master 127.2.180.254:45149 was elected leader, sending a full tablet report...
I20260808 15:02:34.112622 6044 ts_tablet_manager.cc:1428] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Time spent starting tablet: real 0.010s user 0.009s sys 0.002s
I20260808 15:02:34.113621 6045 raft_consensus.cc:348] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.114024 6045 raft_consensus.cc:374] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.114125 6045 raft_consensus.cc:729] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7069b5af8642afae5dadc19dac2185, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.114707 6045 consensus_queue.cc:260] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.114835 6047 raft_consensus.cc:348] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.115242 6047 raft_consensus.cc:374] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.115343 6047 raft_consensus.cc:729] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba7224d134745559ef0e91280c2496d, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.116051 6047 consensus_queue.cc:260] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.116400 6045 ts_tablet_manager.cc:1428] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260808 15:02:34.116775 5810 heartbeater.cc:503] Master 127.2.180.254:45149 was elected leader, sending a full tablet report...
I20260808 15:02:34.120952 6047 ts_tablet_manager.cc:1428] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Time spent starting tablet: real 0.016s user 0.009s sys 0.007s
W20260808 15:02:34.232547 6019 tablet.cc:2369] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:02:34.250528 5811 tablet.cc:2369] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:02:34.274526 5618 tablet.cc:2369] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:02:34.326606 6054 raft_consensus.cc:482] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:34.326905 6054 raft_consensus.cc:504] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.328603 6054 leader_election.cc:288] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841), bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821)
I20260808 15:02:34.335500 6041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.329050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:38283 (local address 127.2.180.194:39841)
0808 15:02:34.329268 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.329275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.329948 (+ 673us) server_negotiation.cc:407] Connection header received
0808 15:02:34.330120 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.330124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.330183 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.330257 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:34.330953 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.331660 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.333753 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.334554 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.334685 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.334736 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.335102 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.335244 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 15:02:34.335561 6043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.329052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:35661 (local address 127.2.180.195:32821)
0808 15:02:34.329279 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.329285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.329914 (+ 629us) server_negotiation.cc:407] Connection header received
0808 15:02:34.330098 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.330102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.330163 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.330233 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:34.330899 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.331645 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.333764 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.334704 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.334856 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.334924 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.335204 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.335330 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:02:34.335620 6055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.328953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:39841 (local address 127.2.180.193:38283)
0808 15:02:34.329838 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:34.329853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:34.329983 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:34.330285 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:34.330290 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:34.330312 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:34.330800 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.330810 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.331790 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.331794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.333601 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.333614 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.334757 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.334765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.334926 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.334958 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:02:34.335032 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 15:02:34.335119 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":40,"thread_start_us":501,"threads_started":1}
I20260808 15:02:34.335692 6056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.328870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:32821 (local address 127.2.180.193:35661)
0808 15:02:34.329804 (+ 934us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:34.329820 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:34.329957 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:34.330268 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:34.330273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:34.330295 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:34.330760 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.330770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.331777 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.331782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.333603 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.333615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.334908 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.334912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.335040 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.335073 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:02:34.335162 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 15:02:34.335278 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":801,"mutex_wait_us":102,"thread_start_us":611,"threads_started":1}
I20260808 15:02:34.336599 5950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc9253164ac46b68cc02d9b5f26cda3" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc667d285fd7413dab838a30cfaf9103" is_pre_election: true
I20260808 15:02:34.336876 5747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc9253164ac46b68cc02d9b5f26cda3" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185" is_pre_election: true
I20260808 15:02:34.337160 5950 raft_consensus.cc:2454] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 0.
I20260808 15:02:34.337322 5747 raft_consensus.cc:2454] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 0.
I20260808 15:02:34.337739 5453 leader_election.cc:302] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ba7224d134745559ef0e91280c2496d, bc667d285fd7413dab838a30cfaf9103; no voters:
I20260808 15:02:34.338069 6054 raft_consensus.cc:2781] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:34.338213 6054 raft_consensus.cc:482] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:34.338315 6054 raft_consensus.cc:3037] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.340492 6054 raft_consensus.cc:504] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.341013 6054 leader_election.cc:288] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 election: Requested vote from peers 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841), bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821)
I20260808 15:02:34.341361 5747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc9253164ac46b68cc02d9b5f26cda3" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185"
I20260808 15:02:34.341429 5950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afc9253164ac46b68cc02d9b5f26cda3" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc667d285fd7413dab838a30cfaf9103"
I20260808 15:02:34.341547 5747 raft_consensus.cc:3037] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.341584 5950 raft_consensus.cc:3037] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.343641 5950 raft_consensus.cc:2454] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 1.
I20260808 15:02:34.343689 5747 raft_consensus.cc:2454] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 1.
I20260808 15:02:34.344043 5453 leader_election.cc:302] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ba7224d134745559ef0e91280c2496d, bc667d285fd7413dab838a30cfaf9103; no voters:
I20260808 15:02:34.344313 6054 raft_consensus.cc:2781] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:34.344719 6054 raft_consensus.cc:686] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 1 LEADER]: Becoming Leader. State: Replica: 6ba7224d134745559ef0e91280c2496d, State: Running, Role: LEADER
I20260808 15:02:34.345182 6054 consensus_queue.cc:237] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.349485 5313 catalog_manager.cc:5495] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ba7224d134745559ef0e91280c2496d (127.2.180.193). New cstate: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } health_report { overall_health: UNKNOWN } } }
Aug 08 15:02:34 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201352, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:34.454195 5462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.441847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42288 (local address 127.2.180.254:45149)
0808 15:02:34.442004 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.442008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.442554 (+ 546us) server_negotiation.cc:407] Connection header received
0808 15:02:34.442719 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.442723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.442801 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.442880 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:34.444015 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.444732 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.445913 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.446344 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.446448 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.449458 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:34.449463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:34.449466 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:34.449494 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:34.451730 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:34.452399 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:34.452403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:34.452405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:34.452519 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:34.452897 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:34.452901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:34.452903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:34.453224 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:34.453337 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.453755 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.453924 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:02:34.454210 6067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.441921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45149 (local address 127.0.0.1:42288)
0808 15:02:34.442450 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:34.442466 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:34.442612 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:34.442921 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:34.442926 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:34.443599 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:34.443879 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.443888 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.444853 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.444858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.445780 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.445791 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.446510 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.446516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.446667 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.447233 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:34.447250 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:34.449295 (+ 2045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:34.451868 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:34.451881 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:34.451889 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:34.452288 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:34.452630 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:34.452634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:34.452638 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:34.452770 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:34.453329 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:34.453335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:34.453487 (+ 152us) client_negotiation.cc:769] Sending connection context
0808 15:02:34.453634 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:02:34.453814 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":268,"threads_started":1}
I20260808 15:02:34.458711 5313 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:42288:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 15:02:34.459635 5313 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:34.465909 5713 tablet_service.cc:1467] Processing CreateTablet for tablet bcbbe16e1e3d485c801056da8c8ca72f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9fffd62741f4db49c789b7efb23f88f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:02:34.465969 5516 tablet_service.cc:1467] Processing CreateTablet for tablet bcbbe16e1e3d485c801056da8c8ca72f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9fffd62741f4db49c789b7efb23f88f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:02:34.466128 5912 tablet_service.cc:1467] Processing CreateTablet for tablet bcbbe16e1e3d485c801056da8c8ca72f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9fffd62741f4db49c789b7efb23f88f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:02:34.466843 5713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbbe16e1e3d485c801056da8c8ca72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.466863 5516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbbe16e1e3d485c801056da8c8ca72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.466863 5912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bcbbe16e1e3d485c801056da8c8ca72f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.472056 6045 tablet_bootstrap.cc:492] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: Bootstrap starting.
I20260808 15:02:34.473477 6045 tablet_bootstrap.cc:654] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.473666 6044 tablet_bootstrap.cc:492] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: Bootstrap starting.
I20260808 15:02:34.473939 6047 tablet_bootstrap.cc:492] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: Bootstrap starting.
I20260808 15:02:34.475363 6044 tablet_bootstrap.cc:654] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.475395 6047 tablet_bootstrap.cc:654] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.476222 6045 tablet_bootstrap.cc:492] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: No bootstrap required, opened a new log
I20260808 15:02:34.476433 6045 ts_tablet_manager.cc:1397] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 15:02:34.477321 6045 raft_consensus.cc:348] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.477594 6045 raft_consensus.cc:374] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.477926 6045 raft_consensus.cc:729] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7069b5af8642afae5dadc19dac2185, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.477949 6044 tablet_bootstrap.cc:492] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: No bootstrap required, opened a new log
I20260808 15:02:34.477991 6047 tablet_bootstrap.cc:492] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: No bootstrap required, opened a new log
I20260808 15:02:34.478083 6044 ts_tablet_manager.cc:1397] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 15:02:34.478116 6047 ts_tablet_manager.cc:1397] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260808 15:02:34.478204 6045 consensus_queue.cc:260] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.478754 6045 ts_tablet_manager.cc:1428] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:02:34.478695 6044 raft_consensus.cc:348] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.478890 6044 raft_consensus.cc:374] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.478989 6044 raft_consensus.cc:729] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc667d285fd7413dab838a30cfaf9103, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.478844 6047 raft_consensus.cc:348] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.479080 6047 raft_consensus.cc:374] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.479174 6047 raft_consensus.cc:729] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba7224d134745559ef0e91280c2496d, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.479235 6044 consensus_queue.cc:260] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.479147 6052 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.479382 6047 consensus_queue.cc:260] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.479776 6044 ts_tablet_manager.cc:1428] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:02:34.479678 6051 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.479976 6047 ts_tablet_manager.cc:1428] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 15:02:34.480002 6051 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:34.480901 6054 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.481182 6054 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:34.481957 6052 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:34.515472 6054 raft_consensus.cc:482] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:34.515623 6054 raft_consensus.cc:504] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.516227 6054 leader_election.cc:288] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841), bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821)
I20260808 15:02:34.516615 5747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbbe16e1e3d485c801056da8c8ca72f" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185" is_pre_election: true
I20260808 15:02:34.516700 5950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbbe16e1e3d485c801056da8c8ca72f" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc667d285fd7413dab838a30cfaf9103" is_pre_election: true
I20260808 15:02:34.516824 5747 raft_consensus.cc:2454] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 0.
I20260808 15:02:34.516953 5950 raft_consensus.cc:2454] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 0.
I20260808 15:02:34.517231 5445 leader_election.cc:302] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7069b5af8642afae5dadc19dac2185, 6ba7224d134745559ef0e91280c2496d; no voters:
I20260808 15:02:34.517501 6054 raft_consensus.cc:2781] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:34.517597 6054 raft_consensus.cc:482] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:34.517691 6054 raft_consensus.cc:3037] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.519574 6054 raft_consensus.cc:504] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.520072 6054 leader_election.cc:288] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 election: Requested vote from peers 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841), bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821)
I20260808 15:02:34.520413 5747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbbe16e1e3d485c801056da8c8ca72f" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185"
I20260808 15:02:34.520632 5747 raft_consensus.cc:3037] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.520581 5950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bcbbe16e1e3d485c801056da8c8ca72f" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc667d285fd7413dab838a30cfaf9103"
I20260808 15:02:34.520785 5950 raft_consensus.cc:3037] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.523425 5747 raft_consensus.cc:2454] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 1.
I20260808 15:02:34.523463 5950 raft_consensus.cc:2454] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 1.
I20260808 15:02:34.523847 5453 leader_election.cc:302] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5f7069b5af8642afae5dadc19dac2185, 6ba7224d134745559ef0e91280c2496d, bc667d285fd7413dab838a30cfaf9103; no voters:
I20260808 15:02:34.524127 6054 raft_consensus.cc:2781] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:34.524287 6054 raft_consensus.cc:686] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [term 1 LEADER]: Becoming Leader. State: Replica: 6ba7224d134745559ef0e91280c2496d, State: Running, Role: LEADER
I20260808 15:02:34.524524 6054 consensus_queue.cc:237] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.526193 6057 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: TxnStatusTablet state changed. Reason: New leader 6ba7224d134745559ef0e91280c2496d. Latest consensus state: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.526564 6057 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:02:34.527210 6073 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 15:02:34.527498 6073 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 15:02:34.527350 5310 catalog_manager.cc:5495] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ba7224d134745559ef0e91280c2496d (127.2.180.193). New cstate: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } health_report { overall_health: UNKNOWN } } }
I20260808 15:02:34.551323 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.545624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.2.180.193:34459)
0808 15:02:34.545800 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.545806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.545875 (+ 69us) server_negotiation.cc:407] Connection header received
0808 15:02:34.546142 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.546145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.546228 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.546341 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:34.546731 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.547458 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.549319 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.549730 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.549819 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.550382 (+ 563us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:34.550830 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:34.550934 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.551064 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.551127 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:02:34.551512 6067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.545538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34459 (local address 127.0.0.1:39384)
0808 15:02:34.545780 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:34.545800 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:34.545958 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:34.546384 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:34.546389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:34.546403 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:02:34.546607 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.546619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.547590 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.547596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.549188 (+ 1592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.549205 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.549876 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.549883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.550113 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.550183 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:02:34.550935 (+ 752us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:02:34.550956 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 15:02:34.551050 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 15:02:34.551202 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 15:02:34.558403 5950 raft_consensus.cc:1261] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Refusing update from remote peer 6ba7224d134745559ef0e91280c2496d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:34.558408 5747 raft_consensus.cc:1261] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Refusing update from remote peer 6ba7224d134745559ef0e91280c2496d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:34.559371 6057 consensus_queue.cc:1035] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:34.559657 6054 consensus_queue.cc:1035] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:34.575551 6051 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: TxnStatusTablet state changed. Reason: New leader 6ba7224d134745559ef0e91280c2496d. Latest consensus state: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.575910 6052 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: TxnStatusTablet state changed. Reason: New leader 6ba7224d134745559ef0e91280c2496d. Latest consensus state: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.576198 6052 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:34.576905 6051 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:34.580008 6081 mvcc.cc:205] Tried to move back new op lower bound from 7316280748262486016 to 7316280748134744064. Current Snapshot: MvccSnapshot[applied={T|T < 7316280748262486016}]
I20260808 15:02:34.580448 6054 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.580739 6054 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:02:34.582279 6054 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.582507 6054 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P 6ba7224d134745559ef0e91280c2496d: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:02:34.583935 6052 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.584161 6052 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P 5f7069b5af8642afae5dadc19dac2185: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:34.587598 6082 mvcc.cc:205] Tried to move back new op lower bound from 7316280748262486016 to 7316280748134744064. Current Snapshot: MvccSnapshot[applied={T|T < 7316280748262486016}]
I20260808 15:02:34.593423 6051 tablet_replica.cc:439] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } } }
I20260808 15:02:34.593662 6051 tablet_replica.cc:442] T bcbbe16e1e3d485c801056da8c8ca72f P bc667d285fd7413dab838a30cfaf9103: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:02:34.605736 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.594335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39386 (local address 127.2.180.193:34459)
0808 15:02:34.594485 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.594490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.600408 (+ 5918us) server_negotiation.cc:407] Connection header received
0808 15:02:34.600599 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.600603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.600664 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.600741 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:34.601252 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.602127 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.604115 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.604515 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.604623 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.605073 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:34.605189 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:34.605252 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.605382 (+ 130us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.605446 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:02:34.621840 6040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.615773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:44329 (local address 127.2.180.193:34459)
0808 15:02:34.615950 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:34.615956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:34.616577 (+ 621us) server_negotiation.cc:407] Connection header received
0808 15:02:34.616707 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:34.616712 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:34.616796 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:34.616901 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:34.617357 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.618443 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.619889 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.620554 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.620682 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.621084 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:34.621233 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:34.621325 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:34.621434 (+ 109us) server_negotiation.cc:299] Negotiation successful
0808 15:02:34.621559 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:02:34.621841 6095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:34.615850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34459 (local address 127.2.180.193:44329)
0808 15:02:34.616482 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:34.616500 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:34.616612 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:34.616910 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:34.616913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:34.616922 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:02:34.617222 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.617231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.618567 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.618587 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.619748 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:34.619761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:34.620720 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:34.620724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:34.620864 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:34.620916 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:02:34.621327 (+ 411us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:02:34.621335 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:02:34.621394 (+ 59us) client_negotiation.cc:240] Negotiation successful
0808 15:02:34.621515 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":306,"threads_started":1}
I20260808 15:02:34.631148 5746 raft_consensus.cc:1261] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Refusing update from remote peer 6ba7224d134745559ef0e91280c2496d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:34.631547 5950 raft_consensus.cc:1261] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Refusing update from remote peer 6ba7224d134745559ef0e91280c2496d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:34.631676 6057 consensus_queue.cc:1035] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:34.632069 6054 consensus_queue.cc:1035] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:34.693900 5310 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:34.694130 5310 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:34.697261 5310 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: Sending DeleteTablet for 3 replicas of tablet afc9253164ac46b68cc02d9b5f26cda3
I20260808 15:02:34.698243 5515 tablet_service.cc:1514] Processing DeleteTablet for tablet afc9253164ac46b68cc02d9b5f26cda3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:34 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:39376
I20260808 15:02:34.698356 5912 tablet_service.cc:1514] Processing DeleteTablet for tablet afc9253164ac46b68cc02d9b5f26cda3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:34 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33254
I20260808 15:02:34.699003 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet afc9253164ac46b68cc02d9b5f26cda3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:34 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:37786
I20260808 15:02:34.699911 6107 tablet_replica.cc:330] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: stopping tablet replica
I20260808 15:02:34.700359 6107 raft_consensus.cc:2229] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:34.700763 6107 raft_consensus.cc:2258] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:34.701794 5310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:02:34.702458 5310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:34.702894 6107 ts_tablet_manager.cc:1905] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:34.708534 6105 tablet_replica.cc:330] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: stopping tablet replica
I20260808 15:02:34.708930 6105 raft_consensus.cc:2229] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:34.709293 6105 raft_consensus.cc:2258] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:34.710906 6107 ts_tablet_manager.cc:1918] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:34.711104 6107 log.cc:1196] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/afc9253164ac46b68cc02d9b5f26cda3
I20260808 15:02:34.711727 6107 ts_tablet_manager.cc:1939] T afc9253164ac46b68cc02d9b5f26cda3 P 5f7069b5af8642afae5dadc19dac2185: Deleting consensus metadata
I20260808 15:02:34.712289 5516 tablet_service.cc:1467] Processing CreateTablet for tablet e18f10ad13df4fed8eadaa5aa024452d (DEFAULT_TABLE table=test-table [id=e2d84b9ec95f4222863487840c62b0c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:34.712739 5516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18f10ad13df4fed8eadaa5aa024452d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.713186 6105 ts_tablet_manager.cc:1905] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:34.716130 5290 catalog_manager.cc:4841] TS 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841): tablet afc9253164ac46b68cc02d9b5f26cda3 (table test-table [id=9b1d2a8d7c544f5e9fe2422693474b7c]) successfully deleted
I20260808 15:02:34.716861 6106 tablet_replica.cc:330] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: stopping tablet replica
I20260808 15:02:34.716934 5910 tablet_service.cc:1467] Processing CreateTablet for tablet e18f10ad13df4fed8eadaa5aa024452d (DEFAULT_TABLE table=test-table [id=e2d84b9ec95f4222863487840c62b0c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:34.717406 5910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18f10ad13df4fed8eadaa5aa024452d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.717373 6106 raft_consensus.cc:2229] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:02:34.720168 5713 tablet_service.cc:1467] Processing CreateTablet for tablet e18f10ad13df4fed8eadaa5aa024452d (DEFAULT_TABLE table=test-table [id=e2d84b9ec95f4222863487840c62b0c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:34.720567 5713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e18f10ad13df4fed8eadaa5aa024452d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:34.720816 6106 raft_consensus.cc:2258] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:34.720948 6047 tablet_bootstrap.cc:492] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: Bootstrap starting.
I20260808 15:02:34.722680 6047 tablet_bootstrap.cc:654] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.723604 6105 ts_tablet_manager.cc:1918] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:34.723850 6105 log.cc:1196] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/afc9253164ac46b68cc02d9b5f26cda3
I20260808 15:02:34.724399 6105 ts_tablet_manager.cc:1939] T afc9253164ac46b68cc02d9b5f26cda3 P bc667d285fd7413dab838a30cfaf9103: Deleting consensus metadata
I20260808 15:02:34.725999 5292 catalog_manager.cc:4841] TS bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821): tablet afc9253164ac46b68cc02d9b5f26cda3 (table test-table [id=9b1d2a8d7c544f5e9fe2422693474b7c]) successfully deleted
I20260808 15:02:34.726843 6045 tablet_bootstrap.cc:492] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: Bootstrap starting.
I20260808 15:02:34.728295 6106 ts_tablet_manager.cc:1905] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:34.728479 6045 tablet_bootstrap.cc:654] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.729997 6044 tablet_bootstrap.cc:492] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: Bootstrap starting.
I20260808 15:02:34.731644 6044 tablet_bootstrap.cc:654] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:34.738893 6047 tablet_bootstrap.cc:492] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: No bootstrap required, opened a new log
I20260808 15:02:34.739104 6047 ts_tablet_manager.cc:1397] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.005s
I20260808 15:02:34.739924 6047 raft_consensus.cc:348] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.740197 6047 raft_consensus.cc:374] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.740306 6047 raft_consensus.cc:729] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba7224d134745559ef0e91280c2496d, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.740602 6047 consensus_queue.cc:260] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.741106 6106 ts_tablet_manager.cc:1918] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:34.741269 6047 ts_tablet_manager.cc:1428] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:02:34.741392 6106 log.cc:1196] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/afc9253164ac46b68cc02d9b5f26cda3
I20260808 15:02:34.742033 6106 ts_tablet_manager.cc:1939] T afc9253164ac46b68cc02d9b5f26cda3 P 6ba7224d134745559ef0e91280c2496d: Deleting consensus metadata
I20260808 15:02:34.743389 6045 tablet_bootstrap.cc:492] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: No bootstrap required, opened a new log
I20260808 15:02:34.743556 6045 ts_tablet_manager.cc:1397] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.003s
I20260808 15:02:34.744123 5291 catalog_manager.cc:4841] TS 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459): tablet afc9253164ac46b68cc02d9b5f26cda3 (table test-table [id=9b1d2a8d7c544f5e9fe2422693474b7c]) successfully deleted
I20260808 15:02:34.744341 6045 raft_consensus.cc:348] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.744593 6045 raft_consensus.cc:374] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.744688 6045 raft_consensus.cc:729] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7069b5af8642afae5dadc19dac2185, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.744915 6045 consensus_queue.cc:260] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.746807 6045 ts_tablet_manager.cc:1428] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 15:02:34.748260 6044 tablet_bootstrap.cc:492] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: No bootstrap required, opened a new log
I20260808 15:02:34.748412 6044 ts_tablet_manager.cc:1397] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.000s
I20260808 15:02:34.749089 6044 raft_consensus.cc:348] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.749280 6044 raft_consensus.cc:374] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:34.749356 6044 raft_consensus.cc:729] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc667d285fd7413dab838a30cfaf9103, State: Initialized, Role: FOLLOWER
I20260808 15:02:34.749574 6044 consensus_queue.cc:260] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.750048 6044 ts_tablet_manager.cc:1428] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 15:02:34.764009 6097 raft_consensus.cc:482] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:34.764205 6097 raft_consensus.cc:504] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.764827 6097 leader_election.cc:288] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841), bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821)
I20260808 15:02:34.765241 5745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18f10ad13df4fed8eadaa5aa024452d" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185" is_pre_election: true
I20260808 15:02:34.765488 5745 raft_consensus.cc:2454] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 0.
I20260808 15:02:34.765357 5949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18f10ad13df4fed8eadaa5aa024452d" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc667d285fd7413dab838a30cfaf9103" is_pre_election: true
I20260808 15:02:34.765641 5949 raft_consensus.cc:2454] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 0.
I20260808 15:02:34.765873 5445 leader_election.cc:302] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7069b5af8642afae5dadc19dac2185, 6ba7224d134745559ef0e91280c2496d; no voters:
I20260808 15:02:34.766165 6097 raft_consensus.cc:2781] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:34.766305 6097 raft_consensus.cc:482] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:34.766397 6097 raft_consensus.cc:3037] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.769330 6097 raft_consensus.cc:504] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.769980 6097 leader_election.cc:288] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 election: Requested vote from peers 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841), bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821)
I20260808 15:02:34.770366 5745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18f10ad13df4fed8eadaa5aa024452d" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185"
I20260808 15:02:34.770424 5949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e18f10ad13df4fed8eadaa5aa024452d" candidate_uuid: "6ba7224d134745559ef0e91280c2496d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc667d285fd7413dab838a30cfaf9103"
I20260808 15:02:34.770565 5745 raft_consensus.cc:3037] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.770587 5949 raft_consensus.cc:3037] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:34.772701 5949 raft_consensus.cc:2454] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 1.
I20260808 15:02:34.773114 5453 leader_election.cc:302] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ba7224d134745559ef0e91280c2496d, bc667d285fd7413dab838a30cfaf9103; no voters:
I20260808 15:02:34.773409 6097 raft_consensus.cc:2781] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:34.773494 5745 raft_consensus.cc:2454] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ba7224d134745559ef0e91280c2496d in term 1.
I20260808 15:02:34.773595 6097 raft_consensus.cc:686] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 1 LEADER]: Becoming Leader. State: Replica: 6ba7224d134745559ef0e91280c2496d, State: Running, Role: LEADER
I20260808 15:02:34.774019 6097 consensus_queue.cc:237] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:34.776561 5309 catalog_manager.cc:5495] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ba7224d134745559ef0e91280c2496d (127.2.180.193). New cstate: current_term: 1 leader_uuid: "6ba7224d134745559ef0e91280c2496d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } health_report { overall_health: UNKNOWN } } }
I20260808 15:02:34.802996 5949 raft_consensus.cc:1261] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Refusing update from remote peer 6ba7224d134745559ef0e91280c2496d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:34.803058 5745 raft_consensus.cc:1261] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Refusing update from remote peer 6ba7224d134745559ef0e91280c2496d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:34.803558 6097 consensus_queue.cc:1035] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:34.803848 6054 consensus_queue.cc:1035] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:34.808665 6080 mvcc.cc:205] Tried to move back new op lower bound from 7316280749268484096 to 7316280749156339712. Current Snapshot: MvccSnapshot[applied={T|T < 7316280749268484096}]
W20260808 15:02:34.828433 6080 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 15:02:36.422464 5309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:36.422771 5309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:36.427017 5309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: Sending DeleteTablet for 3 replicas of tablet e18f10ad13df4fed8eadaa5aa024452d
I20260808 15:02:36.427695 5516 tablet_service.cc:1514] Processing DeleteTablet for tablet e18f10ad13df4fed8eadaa5aa024452d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:36 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:39376
I20260808 15:02:36.428061 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet e18f10ad13df4fed8eadaa5aa024452d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:36 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:37786
I20260808 15:02:36.428036 5910 tablet_service.cc:1514] Processing DeleteTablet for tablet e18f10ad13df4fed8eadaa5aa024452d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:36 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33254
I20260808 15:02:36.440464 6160 tablet_replica.cc:330] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: stopping tablet replica
I20260808 15:02:36.440902 6158 tablet_replica.cc:330] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: stopping tablet replica
I20260808 15:02:36.441097 6158 raft_consensus.cc:2229] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:36.441326 6158 raft_consensus.cc:2258] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:36.441725 6157 tablet_replica.cc:330] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: stopping tablet replica
I20260808 15:02:36.441696 6160 raft_consensus.cc:2229] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:36.441926 6157 raft_consensus.cc:2229] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:02:36.441938 6160 raft_consensus.cc:2258] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:36.442296 6157 raft_consensus.cc:2258] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:36.442726 6158 ts_tablet_manager.cc:1905] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:36.443122 6157 ts_tablet_manager.cc:1905] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:36.444600 6160 ts_tablet_manager.cc:1905] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:36.449316 6157 ts_tablet_manager.cc:1918] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:02:36.449359 6158 ts_tablet_manager.cc:1918] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:02:36.449471 6157 log.cc:1196] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/e18f10ad13df4fed8eadaa5aa024452d
I20260808 15:02:36.449486 6158 log.cc:1196] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/e18f10ad13df4fed8eadaa5aa024452d
I20260808 15:02:36.449913 6158 ts_tablet_manager.cc:1939] T e18f10ad13df4fed8eadaa5aa024452d P 5f7069b5af8642afae5dadc19dac2185: Deleting consensus metadata
I20260808 15:02:36.449937 6157 ts_tablet_manager.cc:1939] T e18f10ad13df4fed8eadaa5aa024452d P 6ba7224d134745559ef0e91280c2496d: Deleting consensus metadata
I20260808 15:02:36.450973 5291 catalog_manager.cc:4841] TS 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459): tablet e18f10ad13df4fed8eadaa5aa024452d (table test-table [id=e2d84b9ec95f4222863487840c62b0c2]) successfully deleted
I20260808 15:02:36.451021 5290 catalog_manager.cc:4841] TS 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841): tablet e18f10ad13df4fed8eadaa5aa024452d (table test-table [id=e2d84b9ec95f4222863487840c62b0c2]) successfully deleted
I20260808 15:02:36.451282 6160 ts_tablet_manager.cc:1918] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:02:36.451426 6160 log.cc:1196] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/e18f10ad13df4fed8eadaa5aa024452d
I20260808 15:02:36.451923 6160 ts_tablet_manager.cc:1939] T e18f10ad13df4fed8eadaa5aa024452d P bc667d285fd7413dab838a30cfaf9103: Deleting consensus metadata
I20260808 15:02:36.452817 5292 catalog_manager.cc:4841] TS bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821): tablet e18f10ad13df4fed8eadaa5aa024452d (table test-table [id=e2d84b9ec95f4222863487840c62b0c2]) successfully deleted
Aug 08 15:02:36 dist-test-slave-jlzn krb5kdc[5249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:02:36 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:36.485162 6173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.471732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42294 (local address 127.2.180.254:45149)
0808 15:02:36.472526 (+ 794us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:36.472531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:36.472551 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:02:36.472600 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:36.472604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:36.472670 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:36.472758 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:36.473967 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.474770 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.475980 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.476419 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.476545 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.480295 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:36.480300 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:36.480304 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:36.480336 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:36.482677 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:36.483289 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:36.483293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:36.483295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:36.483439 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:36.483785 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:36.483788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:36.483790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:36.484096 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:36.484336 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:36.484606 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 15:02:36.484825 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":541,"threads_started":1}
I20260808 15:02:36.488328 5309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:02:36.488903 5309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:36.494694 5516 tablet_service.cc:1467] Processing CreateTablet for tablet d5b923b911154c7c882c84d79fef36e7 (DEFAULT_TABLE table=test-table [id=64a69cd220fd4daf848377d1dce9f08d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:36.494812 5713 tablet_service.cc:1467] Processing CreateTablet for tablet d5b923b911154c7c882c84d79fef36e7 (DEFAULT_TABLE table=test-table [id=64a69cd220fd4daf848377d1dce9f08d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:36.495132 5516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b923b911154c7c882c84d79fef36e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:36.495064 5910 tablet_service.cc:1467] Processing CreateTablet for tablet d5b923b911154c7c882c84d79fef36e7 (DEFAULT_TABLE table=test-table [id=64a69cd220fd4daf848377d1dce9f08d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:36.495270 5713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b923b911154c7c882c84d79fef36e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:36.495472 5910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5b923b911154c7c882c84d79fef36e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:36.509539 6177 tablet_bootstrap.cc:492] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: Bootstrap starting.
I20260808 15:02:36.512224 6177 tablet_bootstrap.cc:654] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:36.513160 6176 tablet_bootstrap.cc:492] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: Bootstrap starting.
I20260808 15:02:36.513298 6175 tablet_bootstrap.cc:492] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: Bootstrap starting.
I20260808 15:02:36.514707 6176 tablet_bootstrap.cc:654] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:36.515065 6175 tablet_bootstrap.cc:654] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:36.516739 6176 tablet_bootstrap.cc:492] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: No bootstrap required, opened a new log
I20260808 15:02:36.516929 6176 ts_tablet_manager.cc:1397] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 15:02:36.517695 6176 raft_consensus.cc:348] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.518000 6176 raft_consensus.cc:374] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:36.518122 6176 raft_consensus.cc:729] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc667d285fd7413dab838a30cfaf9103, State: Initialized, Role: FOLLOWER
I20260808 15:02:36.518388 6176 consensus_queue.cc:260] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.523372 6175 tablet_bootstrap.cc:492] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: No bootstrap required, opened a new log
I20260808 15:02:36.523517 6175 ts_tablet_manager.cc:1397] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260808 15:02:36.523628 6176 ts_tablet_manager.cc:1428] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260808 15:02:36.524365 6175 raft_consensus.cc:348] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.524595 6175 raft_consensus.cc:374] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:36.524696 6177 tablet_bootstrap.cc:492] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: No bootstrap required, opened a new log
I20260808 15:02:36.524688 6175 raft_consensus.cc:729] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba7224d134745559ef0e91280c2496d, State: Initialized, Role: FOLLOWER
I20260808 15:02:36.524829 6177 ts_tablet_manager.cc:1397] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20260808 15:02:36.524948 6175 consensus_queue.cc:260] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.525584 6177 raft_consensus.cc:348] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.526188 6175 ts_tablet_manager.cc:1428] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:02:36.526512 6177 raft_consensus.cc:374] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:36.526631 6177 raft_consensus.cc:729] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7069b5af8642afae5dadc19dac2185, State: Initialized, Role: FOLLOWER
I20260808 15:02:36.526922 6177 consensus_queue.cc:260] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.528144 6177 ts_tablet_manager.cc:1428] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260808 15:02:36.573680 6182 raft_consensus.cc:482] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:36.573894 6182 raft_consensus.cc:504] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.575392 6182 leader_election.cc:288] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459), 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841)
I20260808 15:02:36.594551 6184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.575766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:53571 (local address 127.2.180.194:39841)
0808 15:02:36.588404 (+ 12638us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:36.588411 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:36.588434 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:02:36.588501 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:36.588506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:36.588578 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:36.588670 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:36.589491 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.590472 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.592521 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.593604 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.593748 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.593801 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:36.594121 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 15:02:36.594217 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12501,"thread_start_us":12563,"threads_started":1}
I20260808 15:02:36.594614 6185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.575774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:39841 (local address 127.2.180.195:53571)
0808 15:02:36.588138 (+ 12364us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:36.588159 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:36.588292 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:36.588887 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:36.588892 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:36.588934 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:36.589287 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:36.589301 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.590619 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.590625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:36.592349 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:36.592364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.593788 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.593794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:36.593964 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.594000 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:02:36.594096 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 15:02:36.594227 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12221,"mutex_wait_us":1,"thread_start_us":12031,"threads_started":1}
I20260808 15:02:36.595216 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.575792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34459 (local address 127.2.180.195:46665)
0808 15:02:36.589201 (+ 13409us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:36.589216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:36.589331 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:36.589641 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:36.589646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:36.589671 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:36.590036 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:36.590049 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.591395 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.591399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:36.593105 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:36.593120 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.594407 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.594413 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:36.594597 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.594634 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 15:02:36.594724 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 15:02:36.594844 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13276,"mutex_wait_us":1,"thread_start_us":13058,"threads_started":1}
I20260808 15:02:36.595255 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.575890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:46665 (local address 127.2.180.193:34459)
0808 15:02:36.588937 (+ 13047us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:36.588942 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:36.589299 (+ 357us) server_negotiation.cc:407] Connection header received
0808 15:02:36.589472 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:36.589476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:36.589531 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:36.589602 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:36.590177 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.591267 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.593253 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.594189 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.594359 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.594428 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:36.594792 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 15:02:36.594918 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12901,"thread_start_us":12733,"threads_started":1}
I20260808 15:02:36.595271 5747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b923b911154c7c882c84d79fef36e7" candidate_uuid: "bc667d285fd7413dab838a30cfaf9103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185" is_pre_election: true
I20260808 15:02:36.595536 5747 raft_consensus.cc:2454] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc667d285fd7413dab838a30cfaf9103 in term 0.
I20260808 15:02:36.596086 5845 leader_election.cc:302] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7069b5af8642afae5dadc19dac2185, bc667d285fd7413dab838a30cfaf9103; no voters:
I20260808 15:02:36.596086 5552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b923b911154c7c882c84d79fef36e7" candidate_uuid: "bc667d285fd7413dab838a30cfaf9103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba7224d134745559ef0e91280c2496d" is_pre_election: true
I20260808 15:02:36.596503 6182 raft_consensus.cc:2781] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:36.596489 5552 raft_consensus.cc:2454] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc667d285fd7413dab838a30cfaf9103 in term 0.
I20260808 15:02:36.596650 6182 raft_consensus.cc:482] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:36.596752 6182 raft_consensus.cc:3037] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:36.598863 6182 raft_consensus.cc:504] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.599392 6182 leader_election.cc:288] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [CANDIDATE]: Term 1 election: Requested vote from peers 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459), 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841)
I20260808 15:02:36.599861 5747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b923b911154c7c882c84d79fef36e7" candidate_uuid: "bc667d285fd7413dab838a30cfaf9103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185"
I20260808 15:02:36.599912 5552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5b923b911154c7c882c84d79fef36e7" candidate_uuid: "bc667d285fd7413dab838a30cfaf9103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba7224d134745559ef0e91280c2496d"
I20260808 15:02:36.600085 5747 raft_consensus.cc:3037] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:36.600102 5552 raft_consensus.cc:3037] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:36.602182 5747 raft_consensus.cc:2454] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc667d285fd7413dab838a30cfaf9103 in term 1.
I20260808 15:02:36.602274 5552 raft_consensus.cc:2454] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc667d285fd7413dab838a30cfaf9103 in term 1.
I20260808 15:02:36.602520 5845 leader_election.cc:302] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7069b5af8642afae5dadc19dac2185, bc667d285fd7413dab838a30cfaf9103; no voters:
I20260808 15:02:36.602849 6182 raft_consensus.cc:2781] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:36.603319 6182 raft_consensus.cc:686] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 1 LEADER]: Becoming Leader. State: Replica: bc667d285fd7413dab838a30cfaf9103, State: Running, Role: LEADER
I20260808 15:02:36.603662 6182 consensus_queue.cc:237] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.606953 5309 catalog_manager.cc:5495] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc667d285fd7413dab838a30cfaf9103 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "bc667d285fd7413dab838a30cfaf9103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } health_report { overall_health: HEALTHY } } }
I20260808 15:02:36.642334 6195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.636647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.2.180.195:32821)
0808 15:02:36.637184 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:36.637188 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:36.637204 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:02:36.637276 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:36.637280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:36.637350 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:36.637489 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:36.637904 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.638680 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.640037 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.640481 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.640590 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.641037 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:36.641723 (+ 686us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:36.641854 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:36.641946 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 15:02:36.642021 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":284,"threads_started":1}
I20260808 15:02:36.655597 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.650925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:38093 (local address 127.2.180.193:34459)
0808 15:02:36.651100 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:36.651105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:36.651631 (+ 526us) server_negotiation.cc:407] Connection header received
0808 15:02:36.651768 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:36.651772 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:36.651865 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:36.651934 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:36.652325 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.653033 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.654322 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.654705 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.654794 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.655118 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:36.655211 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:36.655273 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:36.655355 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 15:02:36.655410 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:02:36.655701 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.650946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34459 (local address 127.2.180.195:38093)
0808 15:02:36.651559 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:36.651574 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:36.651684 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:36.651962 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:36.651965 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:36.651974 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:02:36.652214 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:36.652221 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.653144 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.653147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:36.654213 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:36.654224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.654831 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.654835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:36.654948 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.654993 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:02:36.655278 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:02:36.655284 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:02:36.655334 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 15:02:36.655423 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":304,"threads_started":1}
I20260808 15:02:36.663127 5747 raft_consensus.cc:1261] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Refusing update from remote peer bc667d285fd7413dab838a30cfaf9103: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:36.663455 5552 raft_consensus.cc:1261] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Refusing update from remote peer bc667d285fd7413dab838a30cfaf9103: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:36.663761 6192 consensus_queue.cc:1035] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:36.664201 6182 consensus_queue.cc:1035] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:36.688479 6195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.683553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:50015 (local address 127.2.180.195:32821)
0808 15:02:36.683682 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:36.683687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:36.684077 (+ 390us) server_negotiation.cc:407] Connection header received
0808 15:02:36.684260 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:36.684265 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:36.684328 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:36.684394 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:36.684827 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.685516 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.686967 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.687536 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.687633 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.687991 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:36.688108 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:36.688183 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:36.688253 (+ 70us) server_negotiation.cc:299] Negotiation successful
0808 15:02:36.688310 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:02:36.688616 6205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:36.683461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:32821 (local address 127.2.180.193:50015)
0808 15:02:36.683992 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:36.684011 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:36.684135 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:36.684421 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:36.684426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:36.684437 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:02:36.684683 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:36.684692 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.685643 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.685657 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:36.686838 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:36.686851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:36.687682 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:36.687687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:36.687830 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:36.687877 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:02:36.688181 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:02:36.688187 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:02:36.688233 (+ 46us) client_negotiation.cc:240] Negotiation successful
0808 15:02:36.688323 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":456,"threads_started":1}
I20260808 15:02:36.718981 5309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:36.719211 5309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:36.722138 5309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: Sending DeleteTablet for 3 replicas of tablet d5b923b911154c7c882c84d79fef36e7
I20260808 15:02:36.722863 5516 tablet_service.cc:1514] Processing DeleteTablet for tablet d5b923b911154c7c882c84d79fef36e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:36 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:39376
I20260808 15:02:36.722970 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet d5b923b911154c7c882c84d79fef36e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:36 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:37786
I20260808 15:02:36.723245 6158 tablet_replica.cc:330] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: stopping tablet replica
I20260808 15:02:36.723201 5910 tablet_service.cc:1514] Processing DeleteTablet for tablet d5b923b911154c7c882c84d79fef36e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:36 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33254
I20260808 15:02:36.723261 6157 tablet_replica.cc:330] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: stopping tablet replica
I20260808 15:02:36.723454 6160 tablet_replica.cc:330] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: stopping tablet replica
I20260808 15:02:36.723445 6157 raft_consensus.cc:2229] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:36.723434 6158 raft_consensus.cc:2229] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:36.723614 6158 raft_consensus.cc:2258] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:36.723623 6157 raft_consensus.cc:2258] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:36.723639 6160 raft_consensus.cc:2229] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:02:36.724116 6160 raft_consensus.cc:2258] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:36.724773 6158 ts_tablet_manager.cc:1905] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:36.724925 6157 ts_tablet_manager.cc:1905] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:36.726732 6160 ts_tablet_manager.cc:1905] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:36.731132 6157 ts_tablet_manager.cc:1918] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:36.731133 6158 ts_tablet_manager.cc:1918] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:36.731272 6158 log.cc:1196] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/d5b923b911154c7c882c84d79fef36e7
I20260808 15:02:36.731271 6157 log.cc:1196] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/d5b923b911154c7c882c84d79fef36e7
I20260808 15:02:36.731676 6157 ts_tablet_manager.cc:1939] T d5b923b911154c7c882c84d79fef36e7 P 6ba7224d134745559ef0e91280c2496d: Deleting consensus metadata
I20260808 15:02:36.731676 6158 ts_tablet_manager.cc:1939] T d5b923b911154c7c882c84d79fef36e7 P 5f7069b5af8642afae5dadc19dac2185: Deleting consensus metadata
I20260808 15:02:36.732332 5309 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 15:02:36.732702 5291 catalog_manager.cc:4841] TS 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459): tablet d5b923b911154c7c882c84d79fef36e7 (table test-table [id=64a69cd220fd4daf848377d1dce9f08d]) successfully deleted
I20260808 15:02:36.732911 5290 catalog_manager.cc:4841] TS 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841): tablet d5b923b911154c7c882c84d79fef36e7 (table test-table [id=64a69cd220fd4daf848377d1dce9f08d]) successfully deleted
W20260808 15:02:36.732962 5309 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:36.735847 6160 ts_tablet_manager.cc:1918] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:02:36.736091 6160 log.cc:1196] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/d5b923b911154c7c882c84d79fef36e7
I20260808 15:02:36.736584 6160 ts_tablet_manager.cc:1939] T d5b923b911154c7c882c84d79fef36e7 P bc667d285fd7413dab838a30cfaf9103: Deleting consensus metadata
I20260808 15:02:36.737669 5292 catalog_manager.cc:4841] TS bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821): tablet d5b923b911154c7c882c84d79fef36e7 (table test-table [id=64a69cd220fd4daf848377d1dce9f08d]) successfully deleted
I20260808 15:02:36.739363 5516 tablet_service.cc:1467] Processing CreateTablet for tablet f31e4652d6f44b18896469b94be1e71a (DEFAULT_TABLE table=test-table [id=cd7be4465742468da14f795a4a91a755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:36.739775 5516 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31e4652d6f44b18896469b94be1e71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:36.740226 5713 tablet_service.cc:1467] Processing CreateTablet for tablet f31e4652d6f44b18896469b94be1e71a (DEFAULT_TABLE table=test-table [id=cd7be4465742468da14f795a4a91a755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:36.740605 5713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31e4652d6f44b18896469b94be1e71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:36.744205 5910 tablet_service.cc:1467] Processing CreateTablet for tablet f31e4652d6f44b18896469b94be1e71a (DEFAULT_TABLE table=test-table [id=cd7be4465742468da14f795a4a91a755]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:36.744589 5910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f31e4652d6f44b18896469b94be1e71a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:36.745618 6175 tablet_bootstrap.cc:492] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: Bootstrap starting.
I20260808 15:02:36.746785 6177 tablet_bootstrap.cc:492] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: Bootstrap starting.
I20260808 15:02:36.747171 6175 tablet_bootstrap.cc:654] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:36.748404 6177 tablet_bootstrap.cc:654] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:36.751015 6176 tablet_bootstrap.cc:492] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: Bootstrap starting.
I20260808 15:02:36.752651 6176 tablet_bootstrap.cc:654] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:36.752991 6175 tablet_bootstrap.cc:492] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: No bootstrap required, opened a new log
I20260808 15:02:36.753150 6175 ts_tablet_manager.cc:1397] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 15:02:36.753618 6177 tablet_bootstrap.cc:492] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: No bootstrap required, opened a new log
I20260808 15:02:36.753837 6177 ts_tablet_manager.cc:1397] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 15:02:36.754000 6175 raft_consensus.cc:348] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.754258 6175 raft_consensus.cc:374] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:36.754405 6175 raft_consensus.cc:729] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ba7224d134745559ef0e91280c2496d, State: Initialized, Role: FOLLOWER
I20260808 15:02:36.754698 6175 consensus_queue.cc:260] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.754789 6177 raft_consensus.cc:348] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.755178 6177 raft_consensus.cc:374] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:36.755302 6177 raft_consensus.cc:729] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f7069b5af8642afae5dadc19dac2185, State: Initialized, Role: FOLLOWER
I20260808 15:02:36.755586 6177 consensus_queue.cc:260] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.756239 6175 ts_tablet_manager.cc:1428] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:02:36.756281 6177 ts_tablet_manager.cc:1428] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:02:36.761646 6176 tablet_bootstrap.cc:492] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: No bootstrap required, opened a new log
I20260808 15:02:36.761826 6176 ts_tablet_manager.cc:1397] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20260808 15:02:36.762578 6176 raft_consensus.cc:348] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.762837 6176 raft_consensus.cc:374] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:36.762931 6176 raft_consensus.cc:729] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc667d285fd7413dab838a30cfaf9103, State: Initialized, Role: FOLLOWER
I20260808 15:02:36.763181 6176 consensus_queue.cc:260] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.763721 6176 ts_tablet_manager.cc:1428] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 15:02:36.774215 6192 raft_consensus.cc:482] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:36.774376 6192 raft_consensus.cc:504] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.775086 6192 leader_election.cc:288] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459), 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841)
I20260808 15:02:36.775550 5552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31e4652d6f44b18896469b94be1e71a" candidate_uuid: "bc667d285fd7413dab838a30cfaf9103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba7224d134745559ef0e91280c2496d" is_pre_election: true
I20260808 15:02:36.775597 5747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31e4652d6f44b18896469b94be1e71a" candidate_uuid: "bc667d285fd7413dab838a30cfaf9103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185" is_pre_election: true
I20260808 15:02:36.775852 5552 raft_consensus.cc:2454] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc667d285fd7413dab838a30cfaf9103 in term 0.
I20260808 15:02:36.775863 5747 raft_consensus.cc:2454] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc667d285fd7413dab838a30cfaf9103 in term 0.
I20260808 15:02:36.776273 5845 leader_election.cc:302] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7069b5af8642afae5dadc19dac2185, bc667d285fd7413dab838a30cfaf9103; no voters:
I20260808 15:02:36.776536 6192 raft_consensus.cc:2781] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:36.776638 6192 raft_consensus.cc:482] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:36.776767 6192 raft_consensus.cc:3037] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:36.778997 6192 raft_consensus.cc:504] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.779573 6192 leader_election.cc:288] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [CANDIDATE]: Term 1 election: Requested vote from peers 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459), 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841)
I20260808 15:02:36.780040 5552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31e4652d6f44b18896469b94be1e71a" candidate_uuid: "bc667d285fd7413dab838a30cfaf9103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ba7224d134745559ef0e91280c2496d"
I20260808 15:02:36.780107 5747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f31e4652d6f44b18896469b94be1e71a" candidate_uuid: "bc667d285fd7413dab838a30cfaf9103" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f7069b5af8642afae5dadc19dac2185"
I20260808 15:02:36.780246 5552 raft_consensus.cc:3037] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:36.780285 5747 raft_consensus.cc:3037] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:36.782301 5747 raft_consensus.cc:2454] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc667d285fd7413dab838a30cfaf9103 in term 1.
I20260808 15:02:36.782716 5845 leader_election.cc:302] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f7069b5af8642afae5dadc19dac2185, bc667d285fd7413dab838a30cfaf9103; no voters:
I20260808 15:02:36.782992 6192 raft_consensus.cc:2781] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:36.783139 5552 raft_consensus.cc:2454] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc667d285fd7413dab838a30cfaf9103 in term 1.
I20260808 15:02:36.783174 6192 raft_consensus.cc:686] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 1 LEADER]: Becoming Leader. State: Replica: bc667d285fd7413dab838a30cfaf9103, State: Running, Role: LEADER
I20260808 15:02:36.783627 6192 consensus_queue.cc:237] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } }
I20260808 15:02:36.786092 5310 catalog_manager.cc:5495] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc667d285fd7413dab838a30cfaf9103 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "bc667d285fd7413dab838a30cfaf9103" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc667d285fd7413dab838a30cfaf9103" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32821 } health_report { overall_health: HEALTHY } } }
I20260808 15:02:36.809614 5552 raft_consensus.cc:1261] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Refusing update from remote peer bc667d285fd7413dab838a30cfaf9103: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:36.809815 5747 raft_consensus.cc:1261] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Refusing update from remote peer bc667d285fd7413dab838a30cfaf9103: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:36.810170 6202 consensus_queue.cc:1035] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ba7224d134745559ef0e91280c2496d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:36.810523 6192 consensus_queue.cc:1035] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f7069b5af8642afae5dadc19dac2185" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:02:36.832254 6126 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 15:02:38.409330 5313 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:38.409534 5313 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:38.413376 5313 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b01dc10bb6894d1a8ce75c3eef3b10c3: Sending DeleteTablet for 3 replicas of tablet f31e4652d6f44b18896469b94be1e71a
I20260808 15:02:38.414039 5516 tablet_service.cc:1514] Processing DeleteTablet for tablet f31e4652d6f44b18896469b94be1e71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:38 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:39376
I20260808 15:02:38.414132 5713 tablet_service.cc:1514] Processing DeleteTablet for tablet f31e4652d6f44b18896469b94be1e71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:38 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:37786
I20260808 15:02:38.414132 5910 tablet_service.cc:1514] Processing DeleteTablet for tablet f31e4652d6f44b18896469b94be1e71a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:38 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33254
I20260808 15:02:38.414928 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 5418
I20260808 15:02:38.426448 6243 tablet_replica.cc:330] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: stopping tablet replica
I20260808 15:02:38.426641 6245 tablet_replica.cc:330] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: stopping tablet replica
I20260808 15:02:38.426656 6243 raft_consensus.cc:2229] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:38.426867 6243 raft_consensus.cc:2258] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:38.426878 6245 raft_consensus.cc:2229] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:02:38.427244 6245 raft_consensus.cc:2258] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:38.428349 6245 ts_tablet_manager.cc:1905] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:38.433468 6243 ts_tablet_manager.cc:1905] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:38.435796 6245 ts_tablet_manager.cc:1918] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:02:38.435989 6245 log.cc:1196] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/f31e4652d6f44b18896469b94be1e71a
I20260808 15:02:38.436547 6245 ts_tablet_manager.cc:1939] T f31e4652d6f44b18896469b94be1e71a P bc667d285fd7413dab838a30cfaf9103: Deleting consensus metadata
I20260808 15:02:38.437722 5292 catalog_manager.cc:4841] TS bc667d285fd7413dab838a30cfaf9103 (127.2.180.195:32821): tablet f31e4652d6f44b18896469b94be1e71a (table test-table [id=cd7be4465742468da14f795a4a91a755]) successfully deleted
I20260808 15:02:38.439051 6244 tablet_replica.cc:330] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: stopping tablet replica
I20260808 15:02:38.439298 6244 raft_consensus.cc:2229] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:38.439574 6244 raft_consensus.cc:2258] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:38.440892 6244 ts_tablet_manager.cc:1905] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:38.441354 6243 ts_tablet_manager.cc:1918] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:02:38.441542 6243 log.cc:1196] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/f31e4652d6f44b18896469b94be1e71a
I20260808 15:02:38.442006 6243 ts_tablet_manager.cc:1939] T f31e4652d6f44b18896469b94be1e71a P 6ba7224d134745559ef0e91280c2496d: Deleting consensus metadata
I20260808 15:02:38.443018 5291 catalog_manager.cc:4841] TS 6ba7224d134745559ef0e91280c2496d (127.2.180.193:34459): tablet f31e4652d6f44b18896469b94be1e71a (table test-table [id=cd7be4465742468da14f795a4a91a755]) successfully deleted
I20260808 15:02:38.446645 6244 ts_tablet_manager.cc:1918] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 15:02:38.446763 6244 log.cc:1196] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/f31e4652d6f44b18896469b94be1e71a
I20260808 15:02:38.447137 6244 ts_tablet_manager.cc:1939] T f31e4652d6f44b18896469b94be1e71a P 5f7069b5af8642afae5dadc19dac2185: Deleting consensus metadata
I20260808 15:02:38.447988 5290 catalog_manager.cc:4841] TS 5f7069b5af8642afae5dadc19dac2185 (127.2.180.194:39841): tablet f31e4652d6f44b18896469b94be1e71a (table test-table [id=cd7be4465742468da14f795a4a91a755]) successfully deleted
Aug 08 15:02:38 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:38.454293 6247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:38.416583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.2.180.193:34459)
0808 15:02:38.438808 (+ 22225us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:38.438814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:38.438838 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:02:38.438890 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:38.438896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:38.438978 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:38.440972 (+ 1994us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:38.441171 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:38.442342 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:38.444217 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:38.444784 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:38.444914 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:38.449421 (+ 4507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:38.449435 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:38.449452 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:38.449482 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:38.452018 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:38.452590 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:38.452596 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:38.452598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:38.452746 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:38.453067 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:38.453070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:38.453072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:38.453422 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:38.453639 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:38.453878 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 15:02:38.454033 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22077,"thread_start_us":21894,"threads_started":1}
W20260808 15:02:38.454886 5607 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39390
W20260808 15:02:38.455439 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 5418: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:38.455647 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 5418
I20260808 15:02:38.486405 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 5621
Aug 08 15:02:38 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:02:38.501096 6248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:38.487061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37788 (local address 127.2.180.194:39841)
0808 15:02:38.487524 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:38.487529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:38.487544 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:02:38.487596 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:38.487600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:38.487665 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:38.487757 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:38.488975 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:38.489715 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:38.490956 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:38.491370 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:38.491488 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:38.495499 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:38.495513 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:38.495528 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:38.495565 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:38.498419 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:38.499057 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:38.499071 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:38.499073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:38.499235 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:38.499581 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:38.499586 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:38.499587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:38.499982 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:38.500283 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:38.500539 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 15:02:38.500765 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":227,"threads_started":1}
W20260808 15:02:38.501785 5804 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37788
W20260808 15:02:38.502393 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 5621: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:38.502518 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 5621
I20260808 15:02:38.530939 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 5822
Aug 08 15:02:38 dist-test-slave-jlzn krb5kdc[5249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201356, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:02:38.545600 6249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:38.531631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.2.180.195:32821)
0808 15:02:38.532140 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:38.532144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:38.532160 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:02:38.532212 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:38.532216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:38.532275 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:38.532371 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:38.533717 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:38.534450 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:38.535666 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:38.536144 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:38.536258 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:38.540309 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:38.540323 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:38.540340 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:38.540373 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:38.542986 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:38.543607 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:38.543615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:38.543617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:38.543741 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:38.544131 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:38.544136 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:38.544139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:38.544497 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:38.544783 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:38.545101 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 15:02:38.545316 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":255,"threads_started":1}
W20260808 15:02:38.546200 6009 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33266
W20260808 15:02:38.546795 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 5822: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:38.546949 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 5822
I20260808 15:02:38.570248 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 5269
I20260808 15:02:38.571308 5376 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:38.685289 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 5269
2026-08-08T15:02:38Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8006 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:38 dist-test-slave-jlzn krb5kdc[6256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:38 dist-test-slave-jlzn krb5kdc[6256](info): set up 2 sockets
Aug 08 15:02:38 dist-test-slave-jlzn krb5kdc[6256](info): commencing operation
krb5kdc: starting...
W20260808 15:02:39.835291 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.003s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:39 dist-test-slave-jlzn krb5kdc[6256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:40.009248 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35587
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:35587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:40.318249 6273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:40.318511 6273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:40.318641 6273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:40.325508 6273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:40.325603 6273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:40.325676 6273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:40.325749 6273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:40.325811 6273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:40.335111 6273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:35587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:40.336310 6273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:40.337531 6273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:40.348686 6279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:40.348724 6280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:40.348752 6283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:40.349493 6273 server_base.cc:1034] running on GCE node
I20260808 15:02:40.350219 6273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:40.351464 6273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:40.352692 6273 hybrid_clock.cc:648] HybridClock initialized: now 1786201360352632 us; error 42 us; skew 500 ppm
I20260808 15:02:40.353004 6273 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:40 dist-test-slave-jlzn krb5kdc[6256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201360, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:40.360651 6273 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:40.362037 6273 webserver.cc:460] Webserver started at http://127.2.180.254:39183/ using document root <none> and password file <none>
I20260808 15:02:40.362669 6273 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:40.362784 6273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:40.363082 6273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:40.365928 6273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf"
format_stamp: "Formatted at 2026-08-08 15:02:40 on dist-test-slave-jlzn"
server_key: "28efed7b7baff072132387633cc2e446"
server_key_iv: "477cda7e67136be6ff9b70325b82b3c3"
server_key_version: "encryptionkey@0"
I20260808 15:02:40.366691 6273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf"
format_stamp: "Formatted at 2026-08-08 15:02:40 on dist-test-slave-jlzn"
server_key: "28efed7b7baff072132387633cc2e446"
server_key_iv: "477cda7e67136be6ff9b70325b82b3c3"
server_key_version: "encryptionkey@0"
I20260808 15:02:40.371578 6273 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 15:02:40.374810 6293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:40.376076 6273 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:02:40.376245 6273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf"
format_stamp: "Formatted at 2026-08-08 15:02:40 on dist-test-slave-jlzn"
server_key: "28efed7b7baff072132387633cc2e446"
server_key_iv: "477cda7e67136be6ff9b70325b82b3c3"
server_key_version: "encryptionkey@0"
I20260808 15:02:40.376420 6273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:40.389842 6273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:40.393446 6273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:40.393693 6273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:40.410286 6273 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:35587
I20260808 15:02:40.410305 6369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:35587 every 8 connection(s)
I20260808 15:02:40.411639 6273 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:40.415118 6370 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:40.422091 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 6273
I20260808 15:02:40.422317 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:40.423071 2771 external_mini_cluster.cc:1428] Setting key 02c5c7515185da583909ad4916e8ce6c
I20260808 15:02:40.425662 6370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf: Bootstrap starting.
I20260808 15:02:40.428679 6370 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:40.429610 6370 log.cc:824] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:40.432206 6370 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf: No bootstrap required, opened a new log
Aug 08 15:02:40 dist-test-slave-jlzn krb5kdc[6256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:40.439364 6370 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35587 } }
I20260808 15:02:40.439682 6370 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:40.439779 6370 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a95ebbf40144d7990fbf0ed5e8c2abf, State: Initialized, Role: FOLLOWER
I20260808 15:02:40.440371 6370 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35587 } }
I20260808 15:02:40.440585 6370 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:40.440714 6370 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:40.440873 6370 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:40.442948 6370 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35587 } }
I20260808 15:02:40.443409 6370 leader_election.cc:302] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a95ebbf40144d7990fbf0ed5e8c2abf; no voters:
I20260808 15:02:40.443972 6370 leader_election.cc:288] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:40.444113 6376 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:40.444607 6374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:40.425143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.2.180.254:35587)
0808 15:02:40.425912 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:40.425924 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:40.425963 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:02:40.427698 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:40.427719 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:40.428096 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:40.428420 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:40.429904 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:40.431259 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:40.432563 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:40.433216 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:40.433375 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:40.437318 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:40.437328 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:40.437342 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:40.437382 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:40.440383 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:40.441079 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:40.441087 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:40.441095 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:40.441268 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:40.441630 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:40.441635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:40.441640 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:40.442173 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:40.442455 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:40.442906 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 15:02:40.443318 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":347,"threads_started":1}
I20260808 15:02:40.445523 6376 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [term 1 LEADER]: Becoming Leader. State: Replica: 0a95ebbf40144d7990fbf0ed5e8c2abf, State: Running, Role: LEADER
I20260808 15:02:40.445997 6370 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:40.446089 6376 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35587 } }
I20260808 15:02:40.449980 6377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35587 } } }
I20260808 15:02:40.450279 6377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:40.450474 6378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a95ebbf40144d7990fbf0ed5e8c2abf. Latest consensus state: current_term: 1 leader_uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a95ebbf40144d7990fbf0ed5e8c2abf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35587 } } }
I20260808 15:02:40.450738 6378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:40.451748 6387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:40.456023 6387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:40.463641 6387 catalog_manager.cc:1347] Generated new cluster ID: d2aff2b3699c4c5fbd43e9b142b82211
I20260808 15:02:40.463738 6387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:40.487047 6387 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:40.488878 6387 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:40.500144 6387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a95ebbf40144d7990fbf0ed5e8c2abf: Generated new TSK 0
I20260808 15:02:40.501092 6387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:02:40.556314 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35587
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:40.874336 6405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:40.874629 6405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:40.874770 6405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:40.882126 6405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:40.882247 6405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:40.882393 6405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:40.891894 6405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:40.893082 6405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:40.894387 6405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:40.906733 6412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:40.906774 6414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:40.906811 6411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:40.907711 6405 server_base.cc:1034] running on GCE node
I20260808 15:02:40.908463 6405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:40.909272 6405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:40.910457 6405 hybrid_clock.cc:648] HybridClock initialized: now 1786201360910402 us; error 50 us; skew 500 ppm
I20260808 15:02:40.910733 6405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:40 dist-test-slave-jlzn krb5kdc[6256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201360, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:40.913764 6405 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:02:40.915047 6405 webserver.cc:460] Webserver started at http://127.2.180.193:36521/ using document root <none> and password file <none>
I20260808 15:02:40.915665 6405 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:40.915778 6405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:40.916080 6405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:40.918740 6405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "76f7a08796ef4eff86f05b21353b56b6"
format_stamp: "Formatted at 2026-08-08 15:02:40 on dist-test-slave-jlzn"
server_key: "5212a349883389f1b0191d383969e41a"
server_key_iv: "9947a32bafdbc313151e133245723cb0"
server_key_version: "encryptionkey@0"
I20260808 15:02:40.919471 6405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "76f7a08796ef4eff86f05b21353b56b6"
format_stamp: "Formatted at 2026-08-08 15:02:40 on dist-test-slave-jlzn"
server_key: "5212a349883389f1b0191d383969e41a"
server_key_iv: "9947a32bafdbc313151e133245723cb0"
server_key_version: "encryptionkey@0"
I20260808 15:02:40.924283 6405 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 15:02:40.927390 6423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:40.928578 6405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:02:40.928726 6405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "76f7a08796ef4eff86f05b21353b56b6"
format_stamp: "Formatted at 2026-08-08 15:02:40 on dist-test-slave-jlzn"
server_key: "5212a349883389f1b0191d383969e41a"
server_key_iv: "9947a32bafdbc313151e133245723cb0"
server_key_version: "encryptionkey@0"
I20260808 15:02:40.928915 6405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:40.948730 6405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:40.952395 6405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:40.952636 6405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:40.953483 6405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:40.954684 6405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:40.954767 6405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:40.954880 6405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:40.954934 6405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:40.984079 6405 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:41061
I20260808 15:02:40.984169 6590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:41061 every 8 connection(s)
I20260808 15:02:40.985323 6405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:40.992725 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 6405
I20260808 15:02:40.992918 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:40.993551 2771 external_mini_cluster.cc:1428] Setting key 78388963a219a3db9a3337121343ce30
Aug 08 15:02:40 dist-test-slave-jlzn krb5kdc[6256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201360, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:41.004762 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:40.987708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:40031 (local address 127.2.180.254:35587)
0808 15:02:40.988144 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:40.988149 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:40.988959 (+ 810us) server_negotiation.cc:407] Connection header received
0808 15:02:40.990739 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:40.990744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:40.990822 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:40.990918 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:40.992404 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:40.993200 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:40.994761 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:40.995178 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:40.995283 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:40.998566 (+ 3283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:40.998572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:40.998577 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:40.998616 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:41.001738 (+ 3122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:41.002579 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:41.002583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:41.002585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:41.002705 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:41.003138 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:41.003141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:41.003142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:41.003401 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:41.003511 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:41.004268 (+ 757us) server_negotiation.cc:299] Negotiation successful
0808 15:02:41.004427 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":238,"threads_started":1}
I20260808 15:02:41.005893 6597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:40.988138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35587 (local address 127.2.180.193:40031)
0808 15:02:40.988806 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:40.988848 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:40.990482 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:40.991169 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:40.991177 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:40.991659 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:40.992238 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:40.992256 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:40.993377 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:40.993383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:40.994618 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:40.994631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:40.995381 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:40.995389 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:40.995609 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:40.996212 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:40.996234 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:40.998343 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:41.001882 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:41.001890 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:41.001905 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:41.002437 (+ 532us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:41.002802 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:41.002806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:41.002809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:41.003046 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:41.003521 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:41.003529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:41.003923 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 15:02:41.004189 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 15:02:41.004640 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":257,"threads_started":1}
I20260808 15:02:41.007294 6592 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35587
I20260808 15:02:41.007570 6592 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:41.008258 6592 heartbeater.cc:511] Master 127.2.180.254:35587 requested a full tablet report, sending...
I20260808 15:02:41.010200 6317 ts_manager.cc:194] Registered new tserver with Master: 76f7a08796ef4eff86f05b21353b56b6 (127.2.180.193:41061)
I20260808 15:02:41.011570 6317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:40031
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:02:41.036393 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35587
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:41.354143 6601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:41.354403 6601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:41.354499 6601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:41.361426 6601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:41.361521 6601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:41.361634 6601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:41.370932 6601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:41.372128 6601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:41.373361 6601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:41.385663 6607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:41.385628 6608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:41.385643 6611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:41.386359 6601 server_base.cc:1034] running on GCE node
I20260808 15:02:41.386914 6601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:41.387740 6601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:41.388939 6601 hybrid_clock.cc:648] HybridClock initialized: now 1786201361388884 us; error 48 us; skew 500 ppm
I20260808 15:02:41.389212 6601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:41 dist-test-slave-jlzn krb5kdc[6256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201361, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:41.392174 6601 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:02:41.393472 6601 webserver.cc:460] Webserver started at http://127.2.180.194:36847/ using document root <none> and password file <none>
I20260808 15:02:41.394109 6601 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:41.394225 6601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:41.394495 6601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:41.397210 6601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "79d8400a199d43af93571a5ed86af489"
format_stamp: "Formatted at 2026-08-08 15:02:41 on dist-test-slave-jlzn"
server_key: "09b98d10d12e187afd6f9e400c009871"
server_key_iv: "4101457cabebbbfef8699f3052c2c7b7"
server_key_version: "encryptionkey@0"
I20260808 15:02:41.397959 6601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "79d8400a199d43af93571a5ed86af489"
format_stamp: "Formatted at 2026-08-08 15:02:41 on dist-test-slave-jlzn"
server_key: "09b98d10d12e187afd6f9e400c009871"
server_key_iv: "4101457cabebbbfef8699f3052c2c7b7"
server_key_version: "encryptionkey@0"
I20260808 15:02:41.402804 6601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 15:02:41.406098 6620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:41.407315 6601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 15:02:41.407459 6601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "79d8400a199d43af93571a5ed86af489"
format_stamp: "Formatted at 2026-08-08 15:02:41 on dist-test-slave-jlzn"
server_key: "09b98d10d12e187afd6f9e400c009871"
server_key_iv: "4101457cabebbbfef8699f3052c2c7b7"
server_key_version: "encryptionkey@0"
I20260808 15:02:41.407629 6601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:41.435618 6601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:41.439642 6601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:41.439958 6601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:41.440893 6601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:41.442143 6601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:41.442217 6601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:41.442332 6601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:41.442378 6601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:41.472218 6601 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:35471
I20260808 15:02:41.472285 6776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:35471 every 8 connection(s)
I20260808 15:02:41.473408 6601 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:41.482935 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 6601
I20260808 15:02:41.483148 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:41.483626 2771 external_mini_cluster.cc:1428] Setting key 2393a73afb043250d745b46a262ab25b
Aug 08 15:02:41 dist-test-slave-jlzn krb5kdc[6256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201361, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:41.493062 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:41.475727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:52957 (local address 127.2.180.254:35587)
0808 15:02:41.475965 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:41.475971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:41.477068 (+ 1097us) server_negotiation.cc:407] Connection header received
0808 15:02:41.478801 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:41.478806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:41.478874 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:41.478960 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:41.480751 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:41.481495 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:41.483414 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:41.483870 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:41.483961 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:41.487681 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:41.487686 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:41.487689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:41.487721 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:41.490211 (+ 2490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:41.490876 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:41.490880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:41.490883 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:41.491018 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:41.491421 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:41.491424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:41.491427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:41.491704 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:41.491865 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:41.492517 (+ 652us) server_negotiation.cc:299] Negotiation successful
0808 15:02:41.492750 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 15:02:41.493990 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:41.476206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35587 (local address 127.2.180.194:52957)
0808 15:02:41.476897 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:41.476943 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:41.478567 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:41.479179 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:41.479187 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:41.479989 (+ 802us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:41.480600 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:41.480618 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:41.481632 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:41.481636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:41.483279 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:41.483295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:41.484086 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:41.484094 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:41.484326 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:41.485067 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:41.485090 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:41.487475 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:41.490338 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:41.490347 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:41.490362 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:41.490767 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:41.491119 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:41.491124 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:41.491128 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:41.491316 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:41.491847 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:41.491855 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:41.492172 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 15:02:41.492407 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 15:02:41.492898 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":257,"threads_started":1}
I20260808 15:02:41.495378 6778 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35587
I20260808 15:02:41.495656 6778 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:41.496351 6778 heartbeater.cc:511] Master 127.2.180.254:35587 requested a full tablet report, sending...
I20260808 15:02:41.497731 6317 ts_manager.cc:194] Registered new tserver with Master: 79d8400a199d43af93571a5ed86af489 (127.2.180.194:35471)
I20260808 15:02:41.498502 6317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:52957
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:02:41.525010 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35587
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:36293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:41.841631 6788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:41.841883 6788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:41.841984 6788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:41.850132 6788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:41.850232 6788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:41.850345 6788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:41.859683 6788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:36293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:41.860901 6788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:41.862161 6788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:41.874497 6797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:41.874575 6795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:41.874601 6800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:41.875334 6788 server_base.cc:1034] running on GCE node
I20260808 15:02:41.875962 6788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:41.876814 6788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:41.878015 6788 hybrid_clock.cc:648] HybridClock initialized: now 1786201361877959 us; error 40 us; skew 500 ppm
I20260808 15:02:41.878298 6788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:41 dist-test-slave-jlzn krb5kdc[6256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201361, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:41.881510 6788 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:02:41.882980 6788 webserver.cc:460] Webserver started at http://127.2.180.195:34107/ using document root <none> and password file <none>
I20260808 15:02:41.883697 6788 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:41.883829 6788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:41.884127 6788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:41.886925 6788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "6bc43a155fb246efb12077e577c2be7e"
format_stamp: "Formatted at 2026-08-08 15:02:41 on dist-test-slave-jlzn"
server_key: "623298132542050a1ce60c67cfcd1511"
server_key_iv: "8c003983db161e85d077d1264ebfcfba"
server_key_version: "encryptionkey@0"
I20260808 15:02:41.887696 6788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "6bc43a155fb246efb12077e577c2be7e"
format_stamp: "Formatted at 2026-08-08 15:02:41 on dist-test-slave-jlzn"
server_key: "623298132542050a1ce60c67cfcd1511"
server_key_iv: "8c003983db161e85d077d1264ebfcfba"
server_key_version: "encryptionkey@0"
I20260808 15:02:41.892567 6788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 15:02:41.895826 6809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:41.897017 6788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:02:41.897169 6788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "6bc43a155fb246efb12077e577c2be7e"
format_stamp: "Formatted at 2026-08-08 15:02:41 on dist-test-slave-jlzn"
server_key: "623298132542050a1ce60c67cfcd1511"
server_key_iv: "8c003983db161e85d077d1264ebfcfba"
server_key_version: "encryptionkey@0"
I20260808 15:02:41.897379 6788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:41.910300 6788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:41.914081 6788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:41.914328 6788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:41.915171 6788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:41.916441 6788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:41.916520 6788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:41.916636 6788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:41.916694 6788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:41.945928 6788 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:46651
I20260808 15:02:41.945984 6975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:46651 every 8 connection(s)
I20260808 15:02:41.947190 6788 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:41.951295 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 6788
I20260808 15:02:41.951470 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:41.952080 2771 external_mini_cluster.cc:1428] Setting key 4818b2390f682f2036cc264de5e73f3b
Aug 08 15:02:41 dist-test-slave-jlzn krb5kdc[6256](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201361, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:41.964982 6596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:41.949469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:39735 (local address 127.2.180.254:35587)
0808 15:02:41.949662 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:41.949666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:41.950730 (+ 1064us) server_negotiation.cc:407] Connection header received
0808 15:02:41.952674 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:41.952678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:41.952746 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:41.952832 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:41.954400 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:41.955339 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:41.956837 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:41.957245 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:41.957345 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:41.960377 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:41.960381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:41.960385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:41.960414 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:41.962590 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:41.963205 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:41.963209 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:41.963211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:41.963317 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:41.963642 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:41.963645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:41.963654 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:41.963937 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:41.964044 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:41.964594 (+ 550us) server_negotiation.cc:299] Negotiation successful
0808 15:02:41.964751 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:02:41.966094 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:41.949877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35587 (local address 127.2.180.195:39735)
0808 15:02:41.950570 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:41.950612 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:41.952408 (+ 1796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:41.953068 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:41.953076 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:41.953618 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:41.954243 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:41.954262 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:41.955470 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:41.955474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:41.956704 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:41.956717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:41.957407 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:41.957411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:41.957590 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:41.958126 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:41.958146 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:41.960199 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:41.962709 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:41.962716 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:41.962726 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:41.963120 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:41.963388 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:41.963392 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:41.963394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:41.963565 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:41.964044 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:41.964049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:41.964322 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 15:02:41.964525 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 15:02:41.964851 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":299,"threads_started":1}
I20260808 15:02:41.967223 6978 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35587
I20260808 15:02:41.967482 6978 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:41.968173 6978 heartbeater.cc:511] Master 127.2.180.254:35587 requested a full tablet report, sending...
I20260808 15:02:41.969463 6317 ts_manager.cc:194] Registered new tserver with Master: 6bc43a155fb246efb12077e577c2be7e (127.2.180.195:46651)
I20260808 15:02:41.970216 6317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:39735
I20260808 15:02:41.978667 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 15:02:42.009567 6596 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:02:42.007324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42052 (local address 127.2.180.254:35587)
0808 15:02:42.007485 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:42.007489 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:42.007947 (+ 458us) server_negotiation.cc:407] Connection header received
0808 15:02:42.008090 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:42.008094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:42.008164 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:42.008249 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:42.009332 (+ 1083us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42052: BlockingRecv error: recv got EOF from 127.0.0.1:42052 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:02:42.013265 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 6405
I20260808 15:02:42.014362 6592 heartbeater.cc:503] Master 127.2.180.254:35587 was elected leader, sending a full tablet report...
W20260808 15:02:42.016139 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 6405: Not authorized: Client connection negotiation failed: client connection to 127.2.180.193:41061: server requires authentication, but client does not have Kerberos credentials available
I20260808 15:02:42.016306 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 6405
I20260808 15:02:42.038269 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 6601
W20260808 15:02:42.040524 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 6601: Not authorized: Client connection negotiation failed: client connection to 127.2.180.194:35471: server requires authentication, but client does not have Kerberos credentials available
I20260808 15:02:42.040671 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 6601
I20260808 15:02:42.056847 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 6788
W20260808 15:02:42.059389 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 6788: Not authorized: Client connection negotiation failed: client connection to 127.2.180.195:46651: server requires authentication, but client does not have Kerberos credentials available
I20260808 15:02:42.059506 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 6788
I20260808 15:02:42.081427 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 6273
I20260808 15:02:42.082511 6363 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:42.183977 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 6273
2026-08-08T15:02:42Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3497 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:42 dist-test-slave-jlzn krb5kdc[7005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:42 dist-test-slave-jlzn krb5kdc[7005](info): set up 2 sockets
Aug 08 15:02:42 dist-test-slave-jlzn krb5kdc[7005](info): commencing operation
krb5kdc: starting...
W20260808 15:02:43.373893 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.005s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:43 dist-test-slave-jlzn krb5kdc[7005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:43.696678 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:39283
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:39283
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:44.132321 7022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:44.132607 7022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:44.132705 7022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:44.139557 7022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:44.139652 7022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:44.139705 7022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:44.139760 7022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:44.139825 7022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:44.149118 7022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:39283
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:39283
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:44.150286 7022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:44.151479 7022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:44.162407 7030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:44.162400 7032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:44.162456 7029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:44.163400 7022 server_base.cc:1034] running on GCE node
I20260808 15:02:44.164193 7022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:44.165418 7022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:44.166630 7022 hybrid_clock.cc:648] HybridClock initialized: now 1786201364166587 us; error 30 us; skew 500 ppm
I20260808 15:02:44.166950 7022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:44 dist-test-slave-jlzn krb5kdc[7005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201364, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:44.169922 7022 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:44.171329 7022 webserver.cc:460] Webserver started at http://127.2.180.254:41373/ using document root <none> and password file <none>
I20260808 15:02:44.172011 7022 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:44.172132 7022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:44.172418 7022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:44.175216 7022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "0d5ca317ff7d475da28039786b3548e9"
format_stamp: "Formatted at 2026-08-08 15:02:44 on dist-test-slave-jlzn"
server_key: "4a8edb4d79bbf5a0e06b8ec00b9f0fbd"
server_key_iv: "0a7a1c4348aabc9889ff80440c31f1db"
server_key_version: "encryptionkey@0"
I20260808 15:02:44.176049 7022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "0d5ca317ff7d475da28039786b3548e9"
format_stamp: "Formatted at 2026-08-08 15:02:44 on dist-test-slave-jlzn"
server_key: "4a8edb4d79bbf5a0e06b8ec00b9f0fbd"
server_key_iv: "0a7a1c4348aabc9889ff80440c31f1db"
server_key_version: "encryptionkey@0"
I20260808 15:02:44.180858 7022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:02:44.184108 7042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:44.185333 7022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:02:44.185495 7022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "0d5ca317ff7d475da28039786b3548e9"
format_stamp: "Formatted at 2026-08-08 15:02:44 on dist-test-slave-jlzn"
server_key: "4a8edb4d79bbf5a0e06b8ec00b9f0fbd"
server_key_iv: "0a7a1c4348aabc9889ff80440c31f1db"
server_key_version: "encryptionkey@0"
I20260808 15:02:44.185707 7022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:44.213889 7022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:44.217490 7022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:44.217769 7022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:44.234484 7022 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:39283
I20260808 15:02:44.234522 7112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:39283 every 8 connection(s)
I20260808 15:02:44.235773 7022 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:44.239199 7114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:44.241081 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 7022
I20260808 15:02:44.241263 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:44.241890 2771 external_mini_cluster.cc:1428] Setting key 60a4f1675391df8aca41a4ea21b52597
I20260808 15:02:44.250394 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9: Bootstrap starting.
I20260808 15:02:44.253230 7114 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:44.254127 7114 log.cc:824] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:02:44 dist-test-slave-jlzn krb5kdc[7005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:44.257050 7114 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9: No bootstrap required, opened a new log
I20260808 15:02:44.263723 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:44.243630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59172 (local address 127.2.180.254:39283)
0808 15:02:44.244396 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:44.244408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:44.244446 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:02:44.246009 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:44.246031 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:44.246374 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:44.246689 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:44.248457 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:44.249743 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:44.251010 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:44.251677 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:44.251881 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:44.255852 (+ 3971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:44.255864 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:44.255881 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:44.255929 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:44.259533 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:44.260191 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:44.260203 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:44.260211 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:44.260410 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:44.260738 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:44.260743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:44.260746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:44.261437 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:44.261727 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:44.262065 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 15:02:44.262478 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":362,"threads_started":1}
I20260808 15:02:44.266395 7114 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d5ca317ff7d475da28039786b3548e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39283 } }
I20260808 15:02:44.266714 7114 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:44.266813 7114 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d5ca317ff7d475da28039786b3548e9, State: Initialized, Role: FOLLOWER
I20260808 15:02:44.267395 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:02:44.267323 7114 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d5ca317ff7d475da28039786b3548e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39283 } }
I20260808 15:02:44.267576 7114 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:44.267699 7114 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:44.267904 7114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:44.270736 7114 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d5ca317ff7d475da28039786b3548e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39283 } }
I20260808 15:02:44.271267 7114 leader_election.cc:302] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d5ca317ff7d475da28039786b3548e9; no voters:
I20260808 15:02:44.271871 7114 leader_election.cc:288] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:44.271977 7119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:44.273124 7119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [term 1 LEADER]: Becoming Leader. State: Replica: 0d5ca317ff7d475da28039786b3548e9, State: Running, Role: LEADER
I20260808 15:02:44.273626 7119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d5ca317ff7d475da28039786b3548e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39283 } }
I20260808 15:02:44.273869 7114 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:44.276377 7124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d5ca317ff7d475da28039786b3548e9. Latest consensus state: current_term: 1 leader_uuid: "0d5ca317ff7d475da28039786b3548e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d5ca317ff7d475da28039786b3548e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39283 } } }
I20260808 15:02:44.276732 7124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:44.277330 7123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d5ca317ff7d475da28039786b3548e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d5ca317ff7d475da28039786b3548e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39283 } } }
I20260808 15:02:44.277652 7123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:44.277666 7134 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:44.293931 7134 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:44.303617 7134 catalog_manager.cc:1347] Generated new cluster ID: eb3402429009465db9ee2a14bca39b99
I20260808 15:02:44.303776 7134 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:44.320847 7134 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:44.322760 7134 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:44.330602 7134 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d5ca317ff7d475da28039786b3548e9: Generated new TSK 0
I20260808 15:02:44.331329 7134 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:02:44.336673 7117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:44.328026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.2.180.254:39283)
0808 15:02:44.328209 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:44.328215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:44.328793 (+ 578us) server_negotiation.cc:407] Connection header received
0808 15:02:44.328973 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:44.328978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:44.329047 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:44.329128 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:44.330340 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:44.331489 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:44.333141 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:44.333843 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:44.333969 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:44.334945 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:44.334952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:44.334957 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:44.335907 (+ 950us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:44.335982 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:44.336017 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:44.336136 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:44.336233 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 15:02:44.336330 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:02:44.343112 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7022
I20260808 15:02:44.344296 7107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:44.429996 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 7022
2026-08-08T15:02:44Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2241 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:44 dist-test-slave-jlzn krb5kdc[7165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:44 dist-test-slave-jlzn krb5kdc[7165](info): set up 2 sockets
Aug 08 15:02:44 dist-test-slave-jlzn krb5kdc[7165](info): commencing operation
krb5kdc: starting...
W20260808 15:02:45.589322 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:45 dist-test-slave-jlzn krb5kdc[7165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:45.764778 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:44553
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:44553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:46.078823 7184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:46.079137 7184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:46.079242 7184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:46.086025 7184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:46.086119 7184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:46.086171 7184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:46.086225 7184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:46.086270 7184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:46.095412 7184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:44553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:44553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:46.096580 7184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:46.097733 7184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:46.108526 7192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:46.108490 7195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:46.108569 7197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:46.109328 7184 server_base.cc:1034] running on GCE node
I20260808 15:02:46.110054 7184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:46.111322 7184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:46.112516 7184 hybrid_clock.cc:648] HybridClock initialized: now 1786201366112468 us; error 33 us; skew 500 ppm
I20260808 15:02:46.112819 7184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:46 dist-test-slave-jlzn krb5kdc[7165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201366, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:46.120141 7184 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:46.121587 7184 webserver.cc:460] Webserver started at http://127.2.180.254:34429/ using document root <none> and password file <none>
I20260808 15:02:46.122195 7184 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:46.122295 7184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:46.122541 7184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:46.125391 7184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "d1960e8839ce4536890611518f612a3c"
format_stamp: "Formatted at 2026-08-08 15:02:46 on dist-test-slave-jlzn"
server_key: "ff2dae414035c656f8a3fc77fdbe5c8d"
server_key_iv: "4831b955f68d3e038a17d0bd52a09742"
server_key_version: "encryptionkey@0"
I20260808 15:02:46.126062 7184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "d1960e8839ce4536890611518f612a3c"
format_stamp: "Formatted at 2026-08-08 15:02:46 on dist-test-slave-jlzn"
server_key: "ff2dae414035c656f8a3fc77fdbe5c8d"
server_key_iv: "4831b955f68d3e038a17d0bd52a09742"
server_key_version: "encryptionkey@0"
I20260808 15:02:46.130821 7184 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:02:46.133957 7207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:46.135162 7184 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:02:46.135301 7184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "d1960e8839ce4536890611518f612a3c"
format_stamp: "Formatted at 2026-08-08 15:02:46 on dist-test-slave-jlzn"
server_key: "ff2dae414035c656f8a3fc77fdbe5c8d"
server_key_iv: "4831b955f68d3e038a17d0bd52a09742"
server_key_version: "encryptionkey@0"
I20260808 15:02:46.135448 7184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:46.153689 7184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:46.157460 7184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:46.157684 7184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:46.174590 7184 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:44553
I20260808 15:02:46.174630 7285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:44553 every 8 connection(s)
I20260808 15:02:46.175853 7184 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:46.179309 7286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:46.180780 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 7184
I20260808 15:02:46.180956 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:46.181593 2771 external_mini_cluster.cc:1428] Setting key d507846b6a1fec7cd289d65dd79476a7
I20260808 15:02:46.190795 7286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c: Bootstrap starting.
I20260808 15:02:46.194398 7286 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c: Neither blocks nor log segments found. Creating new log.
Aug 08 15:02:46 dist-test-slave-jlzn krb5kdc[7165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:46.195482 7286 log.cc:824] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:46.198323 7286 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c: No bootstrap required, opened a new log
I20260808 15:02:46.203284 7289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:46.183404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39054 (local address 127.2.180.254:44553)
0808 15:02:46.184178 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:46.184189 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:46.184236 (+ 47us) server_negotiation.cc:407] Connection header received
0808 15:02:46.185837 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:46.185859 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:46.186203 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:46.186488 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:46.188216 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:46.189495 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:46.191153 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:46.191786 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:46.191951 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:46.196570 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:46.196580 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:46.196602 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:46.196650 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:46.199485 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:46.200257 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:46.200262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:46.200268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:46.200427 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:46.200816 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:46.200819 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:46.200821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:46.201340 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:46.201568 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:46.201984 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 15:02:46.202318 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":348,"threads_started":1}
I20260808 15:02:46.206406 7286 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1960e8839ce4536890611518f612a3c" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44553 } }
I20260808 15:02:46.206725 7286 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:46.206810 7286 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1960e8839ce4536890611518f612a3c, State: Initialized, Role: FOLLOWER
I20260808 15:02:46.207343 7286 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1960e8839ce4536890611518f612a3c" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44553 } }
I20260808 15:02:46.207557 7286 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:46.207681 7286 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:46.207903 7286 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:46.210268 7286 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1960e8839ce4536890611518f612a3c" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44553 } }
I20260808 15:02:46.210716 7286 leader_election.cc:302] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1960e8839ce4536890611518f612a3c; no voters:
I20260808 15:02:46.211233 7286 leader_election.cc:288] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:46.211329 7291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:46.212280 7291 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [term 1 LEADER]: Becoming Leader. State: Replica: d1960e8839ce4536890611518f612a3c, State: Running, Role: LEADER
I20260808 15:02:46.212754 7291 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1960e8839ce4536890611518f612a3c" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44553 } }
I20260808 15:02:46.212993 7286 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:46.215341 7294 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1960e8839ce4536890611518f612a3c. Latest consensus state: current_term: 1 leader_uuid: "d1960e8839ce4536890611518f612a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1960e8839ce4536890611518f612a3c" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44553 } } }
I20260808 15:02:46.215718 7294 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:46.216182 7293 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1960e8839ce4536890611518f612a3c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1960e8839ce4536890611518f612a3c" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44553 } } }
I20260808 15:02:46.216547 7293 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:46.222290 7303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:46.226445 7303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:46.234282 7303 catalog_manager.cc:1347] Generated new cluster ID: b7adbf9b0db040a4b12ca11ca7ed8178
I20260808 15:02:46.234380 7303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:46.246883 7303 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:46.248600 7303 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:46.267797 7303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1960e8839ce4536890611518f612a3c: Generated new TSK 0
I20260808 15:02:46.268795 7303 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:02:46.323082 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44553
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:46.639057 7320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:46.639375 7320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:46.639519 7320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:46.646579 7320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:46.646692 7320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:46.646865 7320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:46.656230 7320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:46.657410 7320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:46.658723 7320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:46.671231 7331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:46.671307 7328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:46.671279 7329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:46.672098 7320 server_base.cc:1034] running on GCE node
I20260808 15:02:46.672657 7320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:46.673518 7320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:46.674717 7320 hybrid_clock.cc:648] HybridClock initialized: now 1786201366674671 us; error 38 us; skew 500 ppm
I20260808 15:02:46.675009 7320 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:46 dist-test-slave-jlzn krb5kdc[7165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201366, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:46.677977 7320 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:02:46.679347 7320 webserver.cc:460] Webserver started at http://127.2.180.193:45427/ using document root <none> and password file <none>
I20260808 15:02:46.680048 7320 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:46.680168 7320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:46.680454 7320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:46.683209 7320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "aed81fd9a8f24464aa494984a0b8b2f0"
format_stamp: "Formatted at 2026-08-08 15:02:46 on dist-test-slave-jlzn"
server_key: "7ef4459d9422dd782bdab10b4c24c17b"
server_key_iv: "8ae7cb1bb15b5c1076d3e3a4bf6dadad"
server_key_version: "encryptionkey@0"
I20260808 15:02:46.684002 7320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "aed81fd9a8f24464aa494984a0b8b2f0"
format_stamp: "Formatted at 2026-08-08 15:02:46 on dist-test-slave-jlzn"
server_key: "7ef4459d9422dd782bdab10b4c24c17b"
server_key_iv: "8ae7cb1bb15b5c1076d3e3a4bf6dadad"
server_key_version: "encryptionkey@0"
I20260808 15:02:46.688778 7320 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:02:46.691987 7339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:46.693171 7320 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:02:46.693323 7320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "aed81fd9a8f24464aa494984a0b8b2f0"
format_stamp: "Formatted at 2026-08-08 15:02:46 on dist-test-slave-jlzn"
server_key: "7ef4459d9422dd782bdab10b4c24c17b"
server_key_iv: "8ae7cb1bb15b5c1076d3e3a4bf6dadad"
server_key_version: "encryptionkey@0"
I20260808 15:02:46.693498 7320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:46.725673 7320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:46.729338 7320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:46.729595 7320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:46.730480 7320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:46.731760 7320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:46.731861 7320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:46.731997 7320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:46.732051 7320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:46.761720 7320 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:38907
I20260808 15:02:46.761757 7494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:38907 every 8 connection(s)
I20260808 15:02:46.762928 7320 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:46.769824 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 7320
I20260808 15:02:46.770004 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:46.770633 2771 external_mini_cluster.cc:1428] Setting key 54de6fb7be08f75201f09b21660eeb51
Aug 08 15:02:46 dist-test-slave-jlzn krb5kdc[7165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201366, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:46.782658 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:46.765249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:40961 (local address 127.2.180.254:44553)
0808 15:02:46.765672 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:46.765677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:46.766463 (+ 786us) server_negotiation.cc:407] Connection header received
0808 15:02:46.768496 (+ 2033us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:46.768503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:46.768582 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:46.768677 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:46.770178 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:46.771000 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:46.772688 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:46.773083 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:46.773178 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:46.776908 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:46.776914 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:46.776919 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:46.776957 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:46.779684 (+ 2727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:46.780550 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:46.780554 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:46.780556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:46.780675 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:46.781132 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:46.781135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:46.781137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:46.781375 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:46.781477 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:46.782243 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 15:02:46.782403 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":217,"threads_started":1}
I20260808 15:02:46.784075 7498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:46.765621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44553 (local address 127.2.180.193:40961)
0808 15:02:46.766306 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:46.766350 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:46.768207 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:46.768923 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:46.768931 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:46.769418 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:46.769956 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:46.769980 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:46.771169 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:46.771173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:46.772548 (+ 1375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:46.772564 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:46.773308 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:46.773315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:46.773587 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:46.774269 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:46.774302 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:46.776697 (+ 2395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:46.779844 (+ 3147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:46.779853 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:46.779868 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:46.780440 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:46.780773 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:46.780787 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:46.780791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:46.781041 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:46.781489 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:46.781497 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:46.781890 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 15:02:46.782161 (+ 271us) client_negotiation.cc:240] Negotiation successful
0808 15:02:46.782647 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":272,"threads_started":1}
I20260808 15:02:46.785511 7495 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44553
I20260808 15:02:46.785772 7495 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:46.786453 7495 heartbeater.cc:511] Master 127.2.180.254:44553 requested a full tablet report, sending...
I20260808 15:02:46.788503 7234 ts_manager.cc:194] Registered new tserver with Master: aed81fd9a8f24464aa494984a0b8b2f0 (127.2.180.193:38907)
I20260808 15:02:46.789898 7234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:40961
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:02:46.814052 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44553
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:47.125599 7503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:47.125882 7503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:47.126019 7503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:47.133095 7503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:47.133205 7503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:47.133356 7503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:47.142652 7503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:47.143879 7503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:47.145123 7503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:47.157178 7514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:47.157227 7508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:47.157259 7509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:47.158226 7503 server_base.cc:1034] running on GCE node
I20260808 15:02:47.158774 7503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:47.159627 7503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:47.160822 7503 hybrid_clock.cc:648] HybridClock initialized: now 1786201367160767 us; error 46 us; skew 500 ppm
I20260808 15:02:47.161101 7503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:47.164337 7503 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:02:47.165596 7503 webserver.cc:460] Webserver started at http://127.2.180.194:46593/ using document root <none> and password file <none>
I20260808 15:02:47.166218 7503 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:47.166340 7503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:47.166626 7503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:47.169509 7503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "86996ecf53ca48328ffef569110f52b8"
format_stamp: "Formatted at 2026-08-08 15:02:47 on dist-test-slave-jlzn"
server_key: "d8842e8514b69e681d85960a70c2c6ac"
server_key_iv: "878288887cea33b9bc7137cb4066e34d"
server_key_version: "encryptionkey@0"
I20260808 15:02:47.170250 7503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "86996ecf53ca48328ffef569110f52b8"
format_stamp: "Formatted at 2026-08-08 15:02:47 on dist-test-slave-jlzn"
server_key: "d8842e8514b69e681d85960a70c2c6ac"
server_key_iv: "878288887cea33b9bc7137cb4066e34d"
server_key_version: "encryptionkey@0"
I20260808 15:02:47.175577 7503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 15:02:47.179678 7531 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:47.181133 7503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:02:47.181293 7503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "86996ecf53ca48328ffef569110f52b8"
format_stamp: "Formatted at 2026-08-08 15:02:47 on dist-test-slave-jlzn"
server_key: "d8842e8514b69e681d85960a70c2c6ac"
server_key_iv: "878288887cea33b9bc7137cb4066e34d"
server_key_version: "encryptionkey@0"
I20260808 15:02:47.181514 7503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:47.204619 7503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:47.208691 7503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:47.208937 7503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:47.209846 7503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:47.211124 7503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:47.211203 7503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:47.211293 7503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:47.211335 7503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:47.241307 7503 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:37609
I20260808 15:02:47.241351 7689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:37609 every 8 connection(s)
I20260808 15:02:47.242559 7503 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:47.248273 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 7503
I20260808 15:02:47.248427 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:47.248865 2771 external_mini_cluster.cc:1428] Setting key f2ae04af3e9cb44237afbc205ae8ec86
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:47.261668 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:47.244862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:35785 (local address 127.2.180.254:44553)
0808 15:02:47.245020 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:47.245024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:47.246088 (+ 1064us) server_negotiation.cc:407] Connection header received
0808 15:02:47.247849 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:47.247855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:47.247971 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:47.248082 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:47.249689 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.250528 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.252121 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.252538 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.252651 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:47.255906 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:47.255911 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:47.255914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:47.255947 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:47.258781 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.259461 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.259467 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.259470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.259632 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.260049 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.260053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.260056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.260366 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:47.260522 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:47.261113 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 15:02:47.261332 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:02:47.262372 7694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:47.245260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44553 (local address 127.2.180.194:35785)
0808 15:02:47.245928 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:47.245967 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:47.247565 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:47.248279 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:47.248287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:47.248847 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:47.249482 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:47.249520 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.250689 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.250695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:47.251973 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:47.251987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.252760 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.252768 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:47.252977 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:47.253590 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:47.253616 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:47.255663 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:47.258905 (+ 3242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:47.258912 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:47.258923 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:47.259344 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:47.259731 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:47.259735 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:47.259737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:47.259944 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:47.260471 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:47.260476 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:47.260761 (+ 285us) client_negotiation.cc:769] Sending connection context
0808 15:02:47.260983 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 15:02:47.261338 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":259,"threads_started":1}
I20260808 15:02:47.263682 7691 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44553
I20260808 15:02:47.263981 7691 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:47.264659 7691 heartbeater.cc:511] Master 127.2.180.254:44553 requested a full tablet report, sending...
I20260808 15:02:47.266023 7234 ts_manager.cc:194] Registered new tserver with Master: 86996ecf53ca48328ffef569110f52b8 (127.2.180.194:37609)
I20260808 15:02:47.266831 7234 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:35785
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:02:47.292336 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44553
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:47.602620 7697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:47.602901 7697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:47.603044 7697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:47.610061 7697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:47.610175 7697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:47.610337 7697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:47.619760 7697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:47.621003 7697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:47.622259 7697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:47.634667 7702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:47.634732 7706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:47.634738 7703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:47.635566 7697 server_base.cc:1034] running on GCE node
I20260808 15:02:47.636111 7697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:47.636916 7697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:47.638096 7697 hybrid_clock.cc:648] HybridClock initialized: now 1786201367638052 us; error 34 us; skew 500 ppm
I20260808 15:02:47.638404 7697 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:47.641461 7697 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:02:47.642742 7697 webserver.cc:460] Webserver started at http://127.2.180.195:36309/ using document root <none> and password file <none>
I20260808 15:02:47.643356 7697 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:47.643471 7697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:47.643755 7697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:47.646536 7697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "6f74019c7cec44059fae4c93dd494d50"
format_stamp: "Formatted at 2026-08-08 15:02:47 on dist-test-slave-jlzn"
server_key: "532427298e99c8a744738d93e0214b85"
server_key_iv: "7089635a5b82b7a355c93ad86520660a"
server_key_version: "encryptionkey@0"
I20260808 15:02:47.647259 7697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "6f74019c7cec44059fae4c93dd494d50"
format_stamp: "Formatted at 2026-08-08 15:02:47 on dist-test-slave-jlzn"
server_key: "532427298e99c8a744738d93e0214b85"
server_key_iv: "7089635a5b82b7a355c93ad86520660a"
server_key_version: "encryptionkey@0"
I20260808 15:02:47.651938 7697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 15:02:47.655045 7718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:47.656289 7697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:02:47.656431 7697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "6f74019c7cec44059fae4c93dd494d50"
format_stamp: "Formatted at 2026-08-08 15:02:47 on dist-test-slave-jlzn"
server_key: "532427298e99c8a744738d93e0214b85"
server_key_iv: "7089635a5b82b7a355c93ad86520660a"
server_key_version: "encryptionkey@0"
I20260808 15:02:47.656596 7697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:47.670719 7697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:47.674681 7697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:47.674933 7697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:47.675797 7697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:47.677048 7697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:47.677124 7697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:47.677248 7697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:47.677307 7697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:47.707024 7697 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:43073
I20260808 15:02:47.707084 7881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:43073 every 8 connection(s)
I20260808 15:02:47.708242 7697 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:47.716593 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 7697
I20260808 15:02:47.716779 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:47.717379 2771 external_mini_cluster.cc:1428] Setting key 790e0d03a4b3e28d6e59a7b9ca0b61af
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:47.726814 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:47.710542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:32879 (local address 127.2.180.254:44553)
0808 15:02:47.710721 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:47.710726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:47.711827 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 15:02:47.713595 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:47.713608 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:47.713691 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:47.713782 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:47.715226 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.716005 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.718140 (+ 2135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.718597 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.718705 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:47.722040 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:47.722045 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:47.722049 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:47.722090 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:47.724490 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.725113 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.725116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.725119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.725227 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.725549 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.725552 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.725554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.725795 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:47.725898 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:47.726416 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 15:02:47.726573 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:02:47.727659 7885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:47.710941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44553 (local address 127.2.180.195:32879)
0808 15:02:47.711658 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:47.711695 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:47.713311 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:47.714019 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:47.714027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:47.714528 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:47.715065 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:47.715082 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.716160 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.716164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:47.717995 (+ 1831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:47.718008 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.718810 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.718819 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:47.719043 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:47.719684 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:47.719706 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:47.721864 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:47.724622 (+ 2758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:47.724628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:47.724638 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:47.725028 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:47.725301 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:47.725304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:47.725307 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:47.725474 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:47.725880 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:47.725884 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:47.726144 (+ 260us) client_negotiation.cc:769] Sending connection context
0808 15:02:47.726348 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 15:02:47.726687 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":264,"threads_started":1}
I20260808 15:02:47.729000 7882 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44553
I20260808 15:02:47.729244 7882 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:47.729903 7882 heartbeater.cc:511] Master 127.2.180.254:44553 requested a full tablet report, sending...
I20260808 15:02:47.731313 7232 ts_manager.cc:194] Registered new tserver with Master: 6f74019c7cec44059fae4c93dd494d50 (127.2.180.195:43073)
I20260808 15:02:47.732342 7232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:32879
I20260808 15:02:47.732520 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:47.780025 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:47.767212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39062 (local address 127.2.180.254:44553)
0808 15:02:47.767430 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:47.767437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:47.767870 (+ 433us) server_negotiation.cc:407] Connection header received
0808 15:02:47.768054 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:47.768058 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:47.768126 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:47.768207 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:47.769663 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.770386 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.771619 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.772048 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.772139 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:47.775605 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:47.775610 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:47.775613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:47.775646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:47.777952 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.778464 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.778468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.778470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.778603 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.778885 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.778887 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.778889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.779177 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:47.779353 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:47.779541 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 15:02:47.779705 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 15:02:47.780529 7232 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39062
I20260808 15:02:47.781987 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7320
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:47.792629 7495 heartbeater.cc:503] Master 127.2.180.254:44553 was elected leader, sending a full tablet report...
I20260808 15:02:47.796494 7900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:47.783014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48628 (local address 127.2.180.193:38907)
0808 15:02:47.783549 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:47.783556 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:47.783580 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:02:47.783637 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:47.783642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:47.783859 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:47.783993 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:47.784953 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.785912 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.787055 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.787618 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.787731 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:47.791121 (+ 3390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:47.791132 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:47.791143 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:47.791175 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:47.793999 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.794577 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.794583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.794585 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.794711 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.795035 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.795038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.795040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.795348 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:47.795627 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:47.795921 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 15:02:47.796162 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":270,"threads_started":1}
W20260808 15:02:47.797638 7489 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:48628
W20260808 15:02:47.798620 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7320: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:47.798816 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 7320
I20260808 15:02:47.818795 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7503
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:02:47.833719 7901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:47.819599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.2.180.194:37609)
0808 15:02:47.820138 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:47.820145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:47.820178 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:02:47.820236 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:47.820242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:47.820447 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:47.820587 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:47.821777 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.822805 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.824230 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.824770 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.824884 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:47.828545 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:47.828555 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:47.828568 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:47.828608 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:47.831083 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.831710 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.831717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.831719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.831873 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.832211 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.832216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.832219 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.832545 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:47.832843 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:47.833127 (+ 284us) server_negotiation.cc:299] Negotiation successful
0808 15:02:47.833392 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":271,"threads_started":1}
W20260808 15:02:47.834864 7684 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56446
W20260808 15:02:47.835860 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7503: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:47.835983 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 7503
I20260808 15:02:47.857180 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7697
Aug 08 15:02:47 dist-test-slave-jlzn krb5kdc[7165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:02:47.873327 7902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:47.858054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.2.180.195:43073)
0808 15:02:47.858739 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:47.858745 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:47.858773 (+ 28us) server_negotiation.cc:407] Connection header received
0808 15:02:47.858834 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:47.858840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:47.859039 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:47.859192 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:47.860567 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.861601 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.862879 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:47.863496 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:47.863621 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:47.867556 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:47.867567 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:47.867580 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:47.867613 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:47.870987 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.871668 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.871674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.871677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.871794 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:47.872108 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:47.872111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:47.872113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:47.872443 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:47.872679 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:47.872858 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 15:02:47.873044 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":357,"threads_started":1}
W20260808 15:02:47.874523 7875 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49556
W20260808 15:02:47.875542 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7697: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:02:47.875692 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 7697
I20260808 15:02:47.894937 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7184
I20260808 15:02:47.896054 7279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:48.000765 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 7184
2026-08-08T15:02:48Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3576 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:48 dist-test-slave-jlzn krb5kdc[7906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:48 dist-test-slave-jlzn krb5kdc[7906](info): set up 2 sockets
Aug 08 15:02:48 dist-test-slave-jlzn krb5kdc[7906](info): commencing operation
krb5kdc: starting...
W20260808 15:02:49.157181 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.001s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:49 dist-test-slave-jlzn krb5kdc[7906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:49.336400 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:42775
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:42775
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:49.644924 7923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:49.645236 7923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:49.645376 7923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:49.652377 7923 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:49.652483 7923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:49.652558 7923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:49.652635 7923 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:49.652694 7923 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:49.662142 7923 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:42775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:42775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:49.663347 7923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:49.664640 7923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:49.675961 7935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:49.676008 7930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:49.675959 7931 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:49.676723 7923 server_base.cc:1034] running on GCE node
I20260808 15:02:49.677429 7923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:49.678699 7923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:49.679883 7923 hybrid_clock.cc:648] HybridClock initialized: now 1786201369679829 us; error 44 us; skew 500 ppm
I20260808 15:02:49.680197 7923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:49 dist-test-slave-jlzn krb5kdc[7906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201369, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:49.683099 7923 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:49.684365 7923 webserver.cc:460] Webserver started at http://127.2.180.254:38183/ using document root <none> and password file <none>
I20260808 15:02:49.684959 7923 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:49.685056 7923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:49.685307 7923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:49.688050 7923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "9253f2a7696d4bcd99b027774731844d"
format_stamp: "Formatted at 2026-08-08 15:02:49 on dist-test-slave-jlzn"
server_key: "b4fd92b68786fc49fe5b8b4b5047b5e7"
server_key_iv: "12f9c9988c6cd1e7521152e7684942df"
server_key_version: "encryptionkey@0"
I20260808 15:02:49.688795 7923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "9253f2a7696d4bcd99b027774731844d"
format_stamp: "Formatted at 2026-08-08 15:02:49 on dist-test-slave-jlzn"
server_key: "b4fd92b68786fc49fe5b8b4b5047b5e7"
server_key_iv: "12f9c9988c6cd1e7521152e7684942df"
server_key_version: "encryptionkey@0"
I20260808 15:02:49.693411 7923 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:02:49.696488 7944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:49.697613 7923 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:02:49.697773 7923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "9253f2a7696d4bcd99b027774731844d"
format_stamp: "Formatted at 2026-08-08 15:02:49 on dist-test-slave-jlzn"
server_key: "b4fd92b68786fc49fe5b8b4b5047b5e7"
server_key_iv: "12f9c9988c6cd1e7521152e7684942df"
server_key_version: "encryptionkey@0"
I20260808 15:02:49.697921 7923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:49.713680 7923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:49.717422 7923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:49.717628 7923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:49.734045 7923 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:42775
I20260808 15:02:49.734086 8019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:42775 every 8 connection(s)
I20260808 15:02:49.735230 7923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:49.738565 8021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:49.739897 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 7923
I20260808 15:02:49.740128 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:49.740940 2771 external_mini_cluster.cc:1428] Setting key 9ed7b89cadacd663d471a1617a6d9fcd
I20260808 15:02:49.750808 8021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d: Bootstrap starting.
Aug 08 15:02:49 dist-test-slave-jlzn krb5kdc[7906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:49.753913 8021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:49.755091 8021 log.cc:824] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:49.757855 8021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d: No bootstrap required, opened a new log
I20260808 15:02:49.761601 8025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:49.742869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36920 (local address 127.2.180.254:42775)
0808 15:02:49.743585 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:49.743595 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:49.743634 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:02:49.745234 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:49.745258 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:49.745598 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:49.745942 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:49.747350 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:49.748687 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:49.749848 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:49.750467 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:49.750625 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:49.754885 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:49.754893 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:49.754904 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:49.754947 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:49.757911 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:49.758553 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:49.758560 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:49.758565 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:49.758732 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:49.759084 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:49.759091 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:49.759093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:49.759580 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:49.759927 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:49.760324 (+ 397us) server_negotiation.cc:299] Negotiation successful
0808 15:02:49.760691 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":320,"threads_started":1}
I20260808 15:02:49.765839 8021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253f2a7696d4bcd99b027774731844d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42775 } }
I20260808 15:02:49.766144 8021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:49.766235 8021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9253f2a7696d4bcd99b027774731844d, State: Initialized, Role: FOLLOWER
I20260808 15:02:49.766753 8021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253f2a7696d4bcd99b027774731844d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42775 } }
I20260808 15:02:49.766965 8021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:49.767093 8021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:49.767241 8021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:49.769335 8021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253f2a7696d4bcd99b027774731844d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42775 } }
I20260808 15:02:49.769788 8021 leader_election.cc:302] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9253f2a7696d4bcd99b027774731844d; no voters:
I20260808 15:02:49.772066 8021 leader_election.cc:288] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:49.772184 8027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:49.772840 8027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [term 1 LEADER]: Becoming Leader. State: Replica: 9253f2a7696d4bcd99b027774731844d, State: Running, Role: LEADER
I20260808 15:02:49.773363 8027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253f2a7696d4bcd99b027774731844d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42775 } }
I20260808 15:02:49.773890 8021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:49.775880 8028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9253f2a7696d4bcd99b027774731844d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253f2a7696d4bcd99b027774731844d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42775 } } }
I20260808 15:02:49.776233 8028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:49.778317 8030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9253f2a7696d4bcd99b027774731844d. Latest consensus state: current_term: 1 leader_uuid: "9253f2a7696d4bcd99b027774731844d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9253f2a7696d4bcd99b027774731844d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42775 } } }
I20260808 15:02:49.778609 8030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:49.780824 8034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:49.784842 8034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:49.792639 8034 catalog_manager.cc:1347] Generated new cluster ID: f91665c2541345859fe9f175771f2c1d
I20260808 15:02:49.792742 8034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:49.801357 8034 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:49.803035 8034 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:49.813747 8034 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9253f2a7696d4bcd99b027774731844d: Generated new TSK 0
I20260808 15:02:49.814584 8034 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:02:49.860514 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42775
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:50.171625 8054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:50.171936 8054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:50.172075 8054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:50.179003 8054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:50.179102 8054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:50.179288 8054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:50.188619 8054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:50.189764 8054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:50.191052 8054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:50.203114 8062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:50.203080 8063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:50.203152 8065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:50.204699 8054 server_base.cc:1034] running on GCE node
I20260808 15:02:50.205282 8054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:50.206128 8054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:50.207324 8054 hybrid_clock.cc:648] HybridClock initialized: now 1786201370207270 us; error 43 us; skew 500 ppm
I20260808 15:02:50.207598 8054 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:50 dist-test-slave-jlzn krb5kdc[7906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201370, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:50.210474 8054 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:02:50.211711 8054 webserver.cc:460] Webserver started at http://127.2.180.193:39953/ using document root <none> and password file <none>
I20260808 15:02:50.212364 8054 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:50.212477 8054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:50.212769 8054 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:50.215523 8054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "ce68a30516184c04b4ef59b5d4b2b9f6"
format_stamp: "Formatted at 2026-08-08 15:02:50 on dist-test-slave-jlzn"
server_key: "50fb7f664184772a8b1df57c5c22744d"
server_key_iv: "8269fd5ffcb6fc4aa7b0e3361b8f0c71"
server_key_version: "encryptionkey@0"
I20260808 15:02:50.216302 8054 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "ce68a30516184c04b4ef59b5d4b2b9f6"
format_stamp: "Formatted at 2026-08-08 15:02:50 on dist-test-slave-jlzn"
server_key: "50fb7f664184772a8b1df57c5c22744d"
server_key_iv: "8269fd5ffcb6fc4aa7b0e3361b8f0c71"
server_key_version: "encryptionkey@0"
I20260808 15:02:50.221031 8054 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:02:50.224115 8076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:50.225292 8054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 15:02:50.225419 8054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "ce68a30516184c04b4ef59b5d4b2b9f6"
format_stamp: "Formatted at 2026-08-08 15:02:50 on dist-test-slave-jlzn"
server_key: "50fb7f664184772a8b1df57c5c22744d"
server_key_iv: "8269fd5ffcb6fc4aa7b0e3361b8f0c71"
server_key_version: "encryptionkey@0"
I20260808 15:02:50.225570 8054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:50.250629 8054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:50.254072 8054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:50.254305 8054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:50.255095 8054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:50.256319 8054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:50.256392 8054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:50.256513 8054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:50.256569 8054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:50.286038 8054 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:38607
I20260808 15:02:50.286094 8243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:38607 every 8 connection(s)
I20260808 15:02:50.287209 8054 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:50.296231 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 8054
I20260808 15:02:50.296434 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:50.297056 2771 external_mini_cluster.cc:1428] Setting key 7ad1554c6bae5d00a137df5676085e67
Aug 08 15:02:50 dist-test-slave-jlzn krb5kdc[7906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201370, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:50.305728 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:50.289525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:40907 (local address 127.2.180.254:42775)
0808 15:02:50.289932 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:50.289936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:50.290790 (+ 854us) server_negotiation.cc:407] Connection header received
0808 15:02:50.292612 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:50.292617 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:50.292685 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:50.292814 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:50.294152 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:50.294893 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:50.296412 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:50.296864 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:50.296975 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:50.300399 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:50.300403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:50.300406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:50.300436 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:50.302896 (+ 2460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:50.303556 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:50.303560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:50.303562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:50.303693 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:50.304111 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:50.304115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:50.304117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:50.304387 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:50.304516 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:50.305146 (+ 630us) server_negotiation.cc:299] Negotiation successful
0808 15:02:50.305362 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":202,"threads_started":1}
I20260808 15:02:50.306618 8247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:50.289992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:42775 (local address 127.2.180.193:40907)
0808 15:02:50.290635 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:50.290688 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:50.292332 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:50.292985 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:50.292992 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:50.293455 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:50.293990 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:50.294008 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:50.295005 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:50.295009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:50.296268 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:50.296287 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:50.297076 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:50.297085 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:50.297304 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:50.297902 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:50.297923 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:50.300204 (+ 2281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:50.303020 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:50.303028 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:50.303043 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:50.303438 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:50.303791 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:50.303796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:50.303818 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:50.303998 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:50.304507 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:50.304515 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:50.304810 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 15:02:50.305024 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 15:02:50.305497 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":246,"threads_started":1}
I20260808 15:02:50.308027 8244 heartbeater.cc:347] Connected to a master server at 127.2.180.254:42775
I20260808 15:02:50.308328 8244 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:50.309158 8244 heartbeater.cc:511] Master 127.2.180.254:42775 requested a full tablet report, sending...
I20260808 15:02:50.311380 7972 ts_manager.cc:194] Registered new tserver with Master: ce68a30516184c04b4ef59b5d4b2b9f6 (127.2.180.193:38607)
I20260808 15:02:50.313405 7972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:40907
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:02:50.341130 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42775
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:50.648064 8250 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:50.648322 8250 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:50.648411 8250 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:50.655210 8250 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:50.655305 8250 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:50.655424 8250 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:50.664429 8250 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:50.665475 8250 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:50.666666 8250 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:50.678750 8256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:50.678689 8257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:50.678715 8263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:50.679512 8250 server_base.cc:1034] running on GCE node
I20260808 15:02:50.680114 8250 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:50.680927 8250 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:50.682113 8250 hybrid_clock.cc:648] HybridClock initialized: now 1786201370682061 us; error 47 us; skew 500 ppm
I20260808 15:02:50.682391 8250 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:50 dist-test-slave-jlzn krb5kdc[7906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201370, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:50.685379 8250 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:02:50.686687 8250 webserver.cc:460] Webserver started at http://127.2.180.194:44665/ using document root <none> and password file <none>
I20260808 15:02:50.687290 8250 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:50.687409 8250 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:50.687700 8250 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:50.690390 8250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "53c1c7aa28534e00bdf584870e826fd1"
format_stamp: "Formatted at 2026-08-08 15:02:50 on dist-test-slave-jlzn"
server_key: "3afa14a9fab327e7bd9b5d9d5e7a463f"
server_key_iv: "549f2810efbb371c27ca4a6cfd461c35"
server_key_version: "encryptionkey@0"
I20260808 15:02:50.691136 8250 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "53c1c7aa28534e00bdf584870e826fd1"
format_stamp: "Formatted at 2026-08-08 15:02:50 on dist-test-slave-jlzn"
server_key: "3afa14a9fab327e7bd9b5d9d5e7a463f"
server_key_iv: "549f2810efbb371c27ca4a6cfd461c35"
server_key_version: "encryptionkey@0"
I20260808 15:02:50.695868 8250 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 15:02:50.698976 8271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:50.700203 8250 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:02:50.700358 8250 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "53c1c7aa28534e00bdf584870e826fd1"
format_stamp: "Formatted at 2026-08-08 15:02:50 on dist-test-slave-jlzn"
server_key: "3afa14a9fab327e7bd9b5d9d5e7a463f"
server_key_iv: "549f2810efbb371c27ca4a6cfd461c35"
server_key_version: "encryptionkey@0"
I20260808 15:02:50.700532 8250 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:50.721076 8250 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:50.725059 8250 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:50.725303 8250 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:50.726172 8250 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:50.727401 8250 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:50.727484 8250 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:50.727612 8250 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:50.727666 8250 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:50.758311 8250 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:33757
I20260808 15:02:50.758368 8443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:33757 every 8 connection(s)
I20260808 15:02:50.759461 8250 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:50.765837 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 8250
I20260808 15:02:50.766029 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:50.766526 2771 external_mini_cluster.cc:1428] Setting key 10d03e83d0990dcd97b177b774506c15
Aug 08 15:02:50 dist-test-slave-jlzn krb5kdc[7906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201370, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:50.779465 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:50.761702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:46713 (local address 127.2.180.254:42775)
0808 15:02:50.761860 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:50.761865 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:50.762890 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 15:02:50.764700 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:50.764706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:50.764794 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:50.764883 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:50.766644 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:50.767363 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:50.769253 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:50.769652 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:50.769746 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:50.773602 (+ 3856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:50.773606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:50.773610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:50.773641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:50.776193 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:50.777191 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:50.777197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:50.777200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:50.777363 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:50.777782 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:50.777787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:50.777791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:50.778124 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:50.778270 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:50.778921 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 15:02:50.779153 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:02:50.780227 8448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:50.762065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:42775 (local address 127.2.180.194:46713)
0808 15:02:50.762733 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:50.762776 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:50.764438 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:50.765110 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:50.765118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:50.765701 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:50.766457 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:50.766482 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:50.767504 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:50.767508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:50.769090 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:50.769106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:50.769874 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:50.769881 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:50.770165 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:50.770809 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:50.770861 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:50.773345 (+ 2484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:50.776318 (+ 2973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:50.776326 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:50.776341 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:50.777052 (+ 711us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:50.777471 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:50.777474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:50.777477 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:50.777673 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:50.778238 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:50.778244 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:50.778540 (+ 296us) client_negotiation.cc:769] Sending connection context
0808 15:02:50.778761 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 15:02:50.779132 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":256,"threads_started":1}
I20260808 15:02:50.781647 8444 heartbeater.cc:347] Connected to a master server at 127.2.180.254:42775
I20260808 15:02:50.781927 8444 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:50.782596 8444 heartbeater.cc:511] Master 127.2.180.254:42775 requested a full tablet report, sending...
I20260808 15:02:50.784022 7972 ts_manager.cc:194] Registered new tserver with Master: 53c1c7aa28534e00bdf584870e826fd1 (127.2.180.194:33757)
I20260808 15:02:50.784826 7972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:46713
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:02:50.810978 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42775
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:51.131181 8454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:51.131489 8454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:51.131660 8454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:51.141664 8454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:51.141786 8454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:51.141945 8454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:51.151386 8454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42775
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:51.152596 8454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:51.153831 8454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:51.166309 8460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:51.166239 8463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:51.166211 8461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:51.166991 8454 server_base.cc:1034] running on GCE node
I20260808 15:02:51.167570 8454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:51.168500 8454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:51.169704 8454 hybrid_clock.cc:648] HybridClock initialized: now 1786201371169649 us; error 46 us; skew 500 ppm
I20260808 15:02:51.169998 8454 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[7906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201371, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:51.176823 8454 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:02:51.178169 8454 webserver.cc:460] Webserver started at http://127.2.180.195:46417/ using document root <none> and password file <none>
I20260808 15:02:51.178782 8454 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:51.178884 8454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:51.179168 8454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:51.181953 8454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "57b1d9421710415f913263ddff631d02"
format_stamp: "Formatted at 2026-08-08 15:02:51 on dist-test-slave-jlzn"
server_key: "54435ef793a84ab27da835f9a17ae8e5"
server_key_iv: "8f9524b28b388d9838129de46e06fb07"
server_key_version: "encryptionkey@0"
I20260808 15:02:51.182706 8454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "57b1d9421710415f913263ddff631d02"
format_stamp: "Formatted at 2026-08-08 15:02:51 on dist-test-slave-jlzn"
server_key: "54435ef793a84ab27da835f9a17ae8e5"
server_key_iv: "8f9524b28b388d9838129de46e06fb07"
server_key_version: "encryptionkey@0"
I20260808 15:02:51.187476 8454 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:02:51.190670 8474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:51.191937 8454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:02:51.192108 8454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "57b1d9421710415f913263ddff631d02"
format_stamp: "Formatted at 2026-08-08 15:02:51 on dist-test-slave-jlzn"
server_key: "54435ef793a84ab27da835f9a17ae8e5"
server_key_iv: "8f9524b28b388d9838129de46e06fb07"
server_key_version: "encryptionkey@0"
I20260808 15:02:51.192313 8454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:51.214895 8454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:51.218339 8454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:51.218569 8454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:51.219380 8454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:51.220665 8454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:51.220742 8454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:51.220866 8454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:51.220920 8454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:51.250356 8454 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:43335
I20260808 15:02:51.250422 8624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:43335 every 8 connection(s)
I20260808 15:02:51.251591 8454 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:51.260231 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 8454
I20260808 15:02:51.260406 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:51.260805 2771 external_mini_cluster.cc:1428] Setting key 7e6974ddb982609857821fd38b50c2cf
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[7906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201371, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:51.269588 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:51.253935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:49715 (local address 127.2.180.254:42775)
0808 15:02:51.254120 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:51.254124 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:51.255190 (+ 1066us) server_negotiation.cc:407] Connection header received
0808 15:02:51.256982 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:51.256988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:51.257079 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:51.257166 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:51.258580 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.259328 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.261002 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.261514 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.261630 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:51.264888 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:51.264892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:51.264896 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:51.264928 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:51.267213 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.267844 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.267849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.267851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.267968 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.268312 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.268315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.268317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.268557 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:51.268671 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:51.269198 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 15:02:51.269352 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:02:51.270519 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:51.254358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:42775 (local address 127.2.180.195:49715)
0808 15:02:51.255030 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:51.255073 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:51.256717 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:51.257398 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:51.257406 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:51.257885 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:51.258408 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:51.258425 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.259475 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.259479 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:51.260788 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:51.260801 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.261734 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.261743 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:51.261968 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:51.262614 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:51.262636 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:51.264708 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:51.267324 (+ 2616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:51.267331 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:51.267341 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:51.267734 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:51.268046 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:51.268049 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:51.268051 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:51.268230 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:51.268644 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:51.268649 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:51.268916 (+ 267us) client_negotiation.cc:769] Sending connection context
0808 15:02:51.269130 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 15:02:51.269497 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":249,"threads_started":1}
I20260808 15:02:51.271718 8626 heartbeater.cc:347] Connected to a master server at 127.2.180.254:42775
I20260808 15:02:51.272017 8626 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:51.272671 8626 heartbeater.cc:511] Master 127.2.180.254:42775 requested a full tablet report, sending...
I20260808 15:02:51.274070 7972 ts_manager.cc:194] Registered new tserver with Master: 57b1d9421710415f913263ddff631d02 (127.2.180.195:43335)
I20260808 15:02:51.275038 7972 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:49715
I20260808 15:02:51.275864 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[7906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260808 15:02:51.316766 8244 heartbeater.cc:503] Master 127.2.180.254:42775 was elected leader, sending a full tablet report...
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[7906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:51.323273 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:51.309379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49356 (local address 127.2.180.193:38607)
0808 15:02:51.309942 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:51.309949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:51.309975 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:02:51.310033 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:51.310039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:51.310249 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:51.310368 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:51.311686 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.312744 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.313886 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.314446 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.314553 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:51.318390 (+ 3837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:51.318402 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:51.318416 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:51.318453 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:51.321039 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.321591 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.321597 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.321599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.321720 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.321993 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.321996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.321998 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.322317 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:51.322531 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:51.322733 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 15:02:51.322920 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":272,"threads_started":1}
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[7906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:51.339710 8246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:51.327289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36932 (local address 127.2.180.254:42775)
0808 15:02:51.327489 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:51.327495 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:51.327645 (+ 150us) server_negotiation.cc:407] Connection header received
0808 15:02:51.327760 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:51.327763 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:51.327876 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:51.327951 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:51.329077 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.329816 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.331039 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.331441 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.331543 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:51.335083 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:51.335087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:51.335090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:51.335125 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:51.337330 (+ 2205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.337928 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.337932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.337934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.338056 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.338380 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.338385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.338388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.338706 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:51.338927 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:51.339213 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 15:02:51.339413 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260808 15:02:51.340253 7971 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36932
I20260808 15:02:51.341158 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 8054
I20260808 15:02:51.352041 8637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:51.341971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49364 (local address 127.2.180.193:38607)
0808 15:02:51.342113 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:51.342117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:51.342305 (+ 188us) server_negotiation.cc:407] Connection header received
0808 15:02:51.342415 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:51.342418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:51.342502 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:51.342577 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:51.343593 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.344318 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.345442 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.345844 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.345935 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:51.347922 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:51.347926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:51.347929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:51.347954 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:51.350125 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.350654 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.350658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.350660 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.350768 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.351027 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.351029 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.351031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.351254 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:51.351416 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:51.351615 (+ 199us) server_negotiation.cc:299] Negotiation successful
0808 15:02:51.351779 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:02:51.352630 8236 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:51.432480 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 8054
I20260808 15:02:51.450233 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 8250
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[7906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:02:51.464929 8645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:51.451046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44190 (local address 127.2.180.194:33757)
0808 15:02:51.451642 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:51.451657 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:51.451684 (+ 27us) server_negotiation.cc:407] Connection header received
0808 15:02:51.451746 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:51.451754 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:51.452002 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:51.452139 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:51.453275 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.454296 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.455451 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.456155 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.456253 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:51.460046 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:51.460058 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:51.460073 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:51.460112 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:51.462686 (+ 2574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.463257 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.463263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.463265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.463385 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.463644 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.463647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.463655 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.464014 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:51.464238 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:51.464446 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 15:02:51.464634 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":298,"threads_started":1}
I20260808 15:02:51.466146 8438 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:51.546665 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 8250
I20260808 15:02:51.568374 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 8454
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[7906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:02:51.583766 8647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:51.569424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42660 (local address 127.2.180.195:43335)
0808 15:02:51.570072 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:51.570079 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:51.570109 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:02:51.570166 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:51.570173 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:51.570374 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:51.570496 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:51.571694 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.572754 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.573879 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:51.574495 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:51.574590 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:51.578723 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:51.578734 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:51.578747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:51.578782 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:51.581431 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.582011 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.582017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.582019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.582141 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:51.582459 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:51.582461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:51.582463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:51.582782 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:51.583007 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:51.583220 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 15:02:51.583414 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":501,"thread_start_us":344,"threads_started":1}
I20260808 15:02:51.584993 8620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:51.664481 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 8454
I20260808 15:02:51.682307 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 7923
I20260808 15:02:51.683432 8014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:51.784915 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 7923
2026-08-08T15:02:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3784 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[8653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[8653](info): set up 2 sockets
Aug 08 15:02:51 dist-test-slave-jlzn krb5kdc[8653](info): commencing operation
krb5kdc: starting...
W20260808 15:02:52.952847 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.006s sys 0.013s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:02:53 dist-test-slave-jlzn krb5kdc[8653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:02:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:02:53.132294 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45345
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45345
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:53.439471 8675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:53.439745 8675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:53.439901 8675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:53.446784 8675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:53.446888 8675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:53.446964 8675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:53.447042 8675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:53.447115 8675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:53.456370 8675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45345
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45345
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:53.457537 8675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:53.458702 8675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:53.469485 8680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:53.469408 8683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:53.469460 8681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:53.470507 8675 server_base.cc:1034] running on GCE node
I20260808 15:02:53.471227 8675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:53.472514 8675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:53.473718 8675 hybrid_clock.cc:648] HybridClock initialized: now 1786201373473673 us; error 40 us; skew 500 ppm
I20260808 15:02:53.474017 8675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:53 dist-test-slave-jlzn krb5kdc[8653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201373, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:53.476971 8675 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:02:53.477965 8675 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:53.478050 8675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:53.478302 8675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:53.481060 8675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "77241f5ece824fabab3054941c8eb9d5"
format_stamp: "Formatted at 2026-08-08 15:02:53 on dist-test-slave-jlzn"
server_key: "bde99976e1f24d51257a5dd58b9ec962"
server_key_iv: "ccee7981f53ee03b8a6f4ea57f93f819"
server_key_version: "encryptionkey@0"
I20260808 15:02:53.481770 8675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "77241f5ece824fabab3054941c8eb9d5"
format_stamp: "Formatted at 2026-08-08 15:02:53 on dist-test-slave-jlzn"
server_key: "bde99976e1f24d51257a5dd58b9ec962"
server_key_iv: "ccee7981f53ee03b8a6f4ea57f93f819"
server_key_version: "encryptionkey@0"
I20260808 15:02:53.486403 8675 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:02:53.489583 8692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:53.490712 8675 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:02:53.490862 8675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "77241f5ece824fabab3054941c8eb9d5"
format_stamp: "Formatted at 2026-08-08 15:02:53 on dist-test-slave-jlzn"
server_key: "bde99976e1f24d51257a5dd58b9ec962"
server_key_iv: "ccee7981f53ee03b8a6f4ea57f93f819"
server_key_version: "encryptionkey@0"
I20260808 15:02:53.491070 8675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:53.502960 8675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:53.506436 8675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:53.506680 8675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:53.522990 8675 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:45345
I20260808 15:02:53.523011 8769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:45345 every 8 connection(s)
I20260808 15:02:53.523769 8675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:53.525240 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 8675
I20260808 15:02:53.525441 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:53.526057 2771 external_mini_cluster.cc:1428] Setting key 97c3b35ccbd8677b0f5077ffa1b4e348
I20260808 15:02:53.527474 8771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 15:02:53 dist-test-slave-jlzn krb5kdc[8653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:53.540592 8771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5: Bootstrap starting.
I20260808 15:02:53.543648 8771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:53.544641 8771 log.cc:824] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:53.547164 8771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5: No bootstrap required, opened a new log
I20260808 15:02:53.547912 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:53.527888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53564 (local address 127.2.180.254:45345)
0808 15:02:53.528736 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:53.528749 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:53.528791 (+ 42us) server_negotiation.cc:407] Connection header received
0808 15:02:53.530384 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:53.530405 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:53.530791 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:53.531116 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:53.532949 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:53.534278 (+ 1329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:53.536042 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:53.536681 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:53.536804 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:53.541223 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:53.541230 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:53.541242 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:53.541279 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:53.544121 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:53.544756 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:53.544764 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:53.544771 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:53.544944 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:53.545316 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:53.545320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:53.545322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:53.545843 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:53.546138 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:53.546538 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 15:02:53.546901 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":347,"threads_started":1}
I20260808 15:02:53.554697 8771 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77241f5ece824fabab3054941c8eb9d5" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45345 } }
I20260808 15:02:53.555037 8771 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:53.555135 8771 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77241f5ece824fabab3054941c8eb9d5, State: Initialized, Role: FOLLOWER
I20260808 15:02:53.555624 8771 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77241f5ece824fabab3054941c8eb9d5" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45345 } }
I20260808 15:02:53.555868 8771 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:53.556007 8771 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:53.556209 8771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:53.558282 8771 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77241f5ece824fabab3054941c8eb9d5" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45345 } }
I20260808 15:02:53.558727 8771 leader_election.cc:302] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77241f5ece824fabab3054941c8eb9d5; no voters:
I20260808 15:02:53.559221 8771 leader_election.cc:288] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:53.559320 8776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:53.560211 8776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [term 1 LEADER]: Becoming Leader. State: Replica: 77241f5ece824fabab3054941c8eb9d5, State: Running, Role: LEADER
I20260808 15:02:53.560737 8776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77241f5ece824fabab3054941c8eb9d5" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45345 } }
I20260808 15:02:53.561203 8771 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:53.563359 8777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77241f5ece824fabab3054941c8eb9d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77241f5ece824fabab3054941c8eb9d5" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45345 } } }
I20260808 15:02:53.563738 8777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:53.564029 8778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77241f5ece824fabab3054941c8eb9d5. Latest consensus state: current_term: 1 leader_uuid: "77241f5ece824fabab3054941c8eb9d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77241f5ece824fabab3054941c8eb9d5" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45345 } } }
I20260808 15:02:53.564395 8778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:53.565795 8790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:53.570080 8790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:53.577968 8790 catalog_manager.cc:1347] Generated new cluster ID: af4ce4543fa24adcb17cb48dc0068a43
I20260808 15:02:53.578069 8790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:53.602308 8790 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:53.604138 8790 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:53.619362 8790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5: Generated new TSK 0
I20260808 15:02:53.620141 8790 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:02:53.676713 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45345
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:53.991097 8805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:53.991362 8805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:53.991487 8805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:53.998405 8805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:53.998520 8805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:53.998669 8805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:54.007787 8805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:54.009020 8805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:54.010242 8805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:54.022330 8818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:54.022390 8815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:54.022396 8816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:54.023149 8805 server_base.cc:1034] running on GCE node
I20260808 15:02:54.023703 8805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:54.024567 8805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:54.025770 8805 hybrid_clock.cc:648] HybridClock initialized: now 1786201374025728 us; error 28 us; skew 500 ppm
I20260808 15:02:54.026062 8805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:54 dist-test-slave-jlzn krb5kdc[8653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201374, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:54.029026 8805 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:02:54.030102 8805 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:54.030193 8805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:54.030444 8805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:54.033234 8805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "ff82246d19de49359140987a4c2405ed"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "689956933247428fcc9131dbc721f74d"
server_key_iv: "017b8c3dc18dd0039ae1e0baa8eef20c"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.033965 8805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "ff82246d19de49359140987a4c2405ed"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "689956933247428fcc9131dbc721f74d"
server_key_iv: "017b8c3dc18dd0039ae1e0baa8eef20c"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.038757 8805 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:02:54.042011 8828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:54.043222 8805 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:02:54.043380 8805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "ff82246d19de49359140987a4c2405ed"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "689956933247428fcc9131dbc721f74d"
server_key_iv: "017b8c3dc18dd0039ae1e0baa8eef20c"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.043578 8805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:54.080245 8805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:54.083968 8805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:54.084200 8805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:54.085100 8805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:54.086339 8805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:54.086427 8805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:54.086542 8805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:54.086615 8805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:54.116806 8805 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:46235
I20260808 15:02:54.116869 8996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:46235 every 8 connection(s)
I20260808 15:02:54.117616 8805 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:54.123910 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 8805
I20260808 15:02:54.124105 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:54.124703 2771 external_mini_cluster.cc:1428] Setting key 42b37cb9186d68a5e6bb1bf1ed0bdd67
Aug 08 15:02:54 dist-test-slave-jlzn krb5kdc[8653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201374, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:54.137406 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:54.119821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:51927 (local address 127.2.180.254:45345)
0808 15:02:54.120220 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:54.120225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:54.121092 (+ 867us) server_negotiation.cc:407] Connection header received
0808 15:02:54.122831 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:54.122837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:54.122914 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:54.123008 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:54.124547 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:54.125327 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:54.127270 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:54.127674 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:54.127766 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:54.131627 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:54.131632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:54.131637 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:54.131675 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:54.134445 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:54.135105 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:54.135109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:54.135111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:54.135228 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:54.135615 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:54.135619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:54.135621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:54.135925 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:54.136056 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:54.136813 (+ 757us) server_negotiation.cc:299] Negotiation successful
0808 15:02:54.137052 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":192,"threads_started":1}
I20260808 15:02:54.137971 9008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:54.120241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45345 (local address 127.2.180.193:51927)
0808 15:02:54.120917 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:54.120962 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:54.122576 (+ 1614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:54.123239 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:54.123246 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:54.123739 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:54.124377 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:54.124396 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:54.125501 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:54.125507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:54.127117 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:54.127133 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:54.127929 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:54.127936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:54.128201 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:54.128877 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:54.128906 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:54.131401 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:54.134569 (+ 3168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:54.134577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:54.134591 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:54.134984 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:54.135329 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:54.135334 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:54.135338 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:54.135502 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:54.136056 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:54.136064 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:54.136344 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 15:02:54.136633 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 15:02:54.136983 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":265,"threads_started":1}
I20260808 15:02:54.139304 9001 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45345
I20260808 15:02:54.139575 9001 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:54.140219 9001 heartbeater.cc:511] Master 127.2.180.254:45345 requested a full tablet report, sending...
I20260808 15:02:54.142223 8712 ts_manager.cc:194] Registered new tserver with Master: ff82246d19de49359140987a4c2405ed (127.2.180.193:46235)
I20260808 15:02:54.143584 8712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:51927
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:02:54.167752 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45345
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:54.470306 9011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:54.470556 9011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:54.470696 9011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:54.477519 9011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:54.477622 9011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:54.477770 9011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:54.486801 9011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:54.487994 9011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:54.489243 9011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:54.501034 9018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:54.501119 9022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:54.501060 9020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:54.502362 9011 server_base.cc:1034] running on GCE node
I20260808 15:02:54.502898 9011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:54.503718 9011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:54.504920 9011 hybrid_clock.cc:648] HybridClock initialized: now 1786201374504863 us; error 46 us; skew 500 ppm
I20260808 15:02:54.505215 9011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:54 dist-test-slave-jlzn krb5kdc[8653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201374, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:54.508168 9011 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:02:54.509192 9011 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:54.509294 9011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:54.509598 9011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:54.512456 9011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "b92e95b1e53f4def9095d8408deb733c"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "60789bf0cbd1e77e56595378e5965f19"
server_key_iv: "b65010e2e535abd5788f6d46ca2afc10"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.513244 9011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "b92e95b1e53f4def9095d8408deb733c"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "60789bf0cbd1e77e56595378e5965f19"
server_key_iv: "b65010e2e535abd5788f6d46ca2afc10"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.518114 9011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 15:02:54.521394 9030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:54.522621 9011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 15:02:54.522766 9011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "b92e95b1e53f4def9095d8408deb733c"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "60789bf0cbd1e77e56595378e5965f19"
server_key_iv: "b65010e2e535abd5788f6d46ca2afc10"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.522964 9011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:54.541751 9011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:54.545307 9011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:54.545552 9011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:54.546438 9011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:54.547740 9011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:54.547863 9011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:54.547971 9011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:54.548045 9011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:54.577777 9011 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:37707
I20260808 15:02:54.577824 9198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:37707 every 8 connection(s)
I20260808 15:02:54.578558 9011 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:54.582558 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 9011
I20260808 15:02:54.582732 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:54.583135 2771 external_mini_cluster.cc:1428] Setting key 4a52b1dae1fbcd547c737952cfbc7533
Aug 08 15:02:54 dist-test-slave-jlzn krb5kdc[8653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201374, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:54.598232 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:54.580921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:33199 (local address 127.2.180.254:45345)
0808 15:02:54.581090 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:54.581094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:54.582129 (+ 1035us) server_negotiation.cc:407] Connection header received
0808 15:02:54.584019 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:54.584025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:54.584116 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:54.584229 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:54.585992 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:54.586702 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:54.588733 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:54.589127 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:54.589225 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:54.592807 (+ 3582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:54.592812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:54.592815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:54.592848 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:54.595339 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:54.596058 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:54.596062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:54.596065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:54.596208 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:54.596601 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:54.596604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:54.596606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:54.596865 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:54.596997 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:54.597654 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 15:02:54.597889 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:02:54.599071 9201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:54.581306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45345 (local address 127.2.180.194:33199)
0808 15:02:54.581988 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:54.582024 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:54.583720 (+ 1696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:54.584419 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:54.584426 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:54.585059 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:54.585816 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:54.585844 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:54.586860 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:54.586866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:54.588605 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:54.588619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:54.589343 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:54.589349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:54.589610 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:54.590253 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:54.590280 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:54.592596 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:54.595459 (+ 2863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:54.595468 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:54.595483 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:54.595945 (+ 462us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:54.596311 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:54.596316 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:54.596320 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:54.596490 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:54.596998 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:54.597005 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:54.597297 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 15:02:54.597517 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 15:02:54.597998 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":293,"threads_started":1}
I20260808 15:02:54.600430 9199 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45345
I20260808 15:02:54.600761 9199 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:54.601517 9199 heartbeater.cc:511] Master 127.2.180.254:45345 requested a full tablet report, sending...
I20260808 15:02:54.603111 8712 ts_manager.cc:194] Registered new tserver with Master: b92e95b1e53f4def9095d8408deb733c (127.2.180.194:37707)
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
I20260808 15:02:54.604111 8712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:33199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:02:54.627051 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45345
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:36479
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:02:54.941488 9205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:54.941738 9205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:54.941838 9205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:54.948693 9205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:54.948786 9205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:54.948896 9205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:54.957938 9205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:36479
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45345
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:54.959025 9205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:54.960211 9205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:54.972332 9215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:54.972289 9218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:54.972365 9210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:54.972903 9205 server_base.cc:1034] running on GCE node
I20260808 15:02:54.973425 9205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:54.974254 9205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:54.975447 9205 hybrid_clock.cc:648] HybridClock initialized: now 1786201374975402 us; error 34 us; skew 500 ppm
I20260808 15:02:54.975733 9205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:02:54 dist-test-slave-jlzn krb5kdc[8653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201374, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:02:54.978601 9205 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:02:54.979597 9205 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:54.979707 9205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:54.980028 9205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:54.982741 9205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "0bbff257a21445abbe19352ae9b29c60"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "f95671a4502be42600e2c9547e916264"
server_key_iv: "4f44ddd7f195298f2c3343aa7e8c8126"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.983500 9205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "0bbff257a21445abbe19352ae9b29c60"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "f95671a4502be42600e2c9547e916264"
server_key_iv: "4f44ddd7f195298f2c3343aa7e8c8126"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.988065 9205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:02:54.991175 9228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:54.992354 9205 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:02:54.992501 9205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "0bbff257a21445abbe19352ae9b29c60"
format_stamp: "Formatted at 2026-08-08 15:02:54 on dist-test-slave-jlzn"
server_key: "f95671a4502be42600e2c9547e916264"
server_key_iv: "4f44ddd7f195298f2c3343aa7e8c8126"
server_key_version: "encryptionkey@0"
I20260808 15:02:54.992702 9205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:55.018536 9205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:55.022082 9205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:55.022323 9205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:55.023168 9205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:55.024391 9205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:55.024487 9205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:55.024585 9205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:55.024654 9205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:55.053818 9205 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:41779
I20260808 15:02:55.053874 9399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:41779 every 8 connection(s)
I20260808 15:02:55.054569 9205 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:55.063249 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 9205
I20260808 15:02:55.063443 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:55.063999 2771 external_mini_cluster.cc:1428] Setting key d37c5b8e7a01ce0c2ac8e37e54bb484e
Aug 08 15:02:55 dist-test-slave-jlzn krb5kdc[8653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201374, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:02:55.073523 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.056789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:46713 (local address 127.2.180.254:45345)
0808 15:02:55.056961 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.056966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.057947 (+ 981us) server_negotiation.cc:407] Connection header received
0808 15:02:55.059653 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.059658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.059743 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.059844 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:55.061291 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.062078 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.063929 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.064481 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.064600 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.068268 (+ 3668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:55.068273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:55.068276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:55.068306 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:55.070603 (+ 2297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.071276 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.071281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.071284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.071397 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.071853 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.071858 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.071862 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.072178 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:55.072315 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.072967 (+ 652us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.073191 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:02:55.074324 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.057169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45345 (local address 127.2.180.195:46713)
0808 15:02:55.057812 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:55.057847 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:55.059408 (+ 1561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:55.060071 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:55.060080 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:55.060556 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:55.061139 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.061157 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.062226 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.062230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.063746 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.063761 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.064709 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.064718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.064956 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.065584 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:02:55.065613 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:55.068074 (+ 2461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:55.070734 (+ 2660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:55.070743 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:55.070756 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:55.071152 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:55.071508 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:02:55.071513 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:02:55.071516 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:02:55.071701 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:02:55.072309 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:55.072316 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:55.072606 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 15:02:55.072846 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 15:02:55.073296 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":262,"threads_started":1}
I20260808 15:02:55.075443 9400 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45345
I20260808 15:02:55.075711 9400 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:55.076313 9400 heartbeater.cc:511] Master 127.2.180.254:45345 requested a full tablet report, sending...
I20260808 15:02:55.077569 8712 ts_manager.cc:194] Registered new tserver with Master: 0bbff257a21445abbe19352ae9b29c60 (127.2.180.195:41779)
I20260808 15:02:55.078469 8712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:46713
I20260808 15:02:55.079454 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:02:55.094875 9006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.083737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53576 (local address 127.2.180.254:45345)
0808 15:02:55.083888 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.083892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.084356 (+ 464us) server_negotiation.cc:407] Connection header received
0808 15:02:55.084510 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.084515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.084569 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.084645 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:55.085989 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.086659 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.087897 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.088289 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.088408 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.090414 (+ 2006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:55.090418 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:55.090421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:55.090449 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:55.092544 (+ 2095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.093120 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.093125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.093127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.093255 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.093595 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.093599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.093601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.093895 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:55.094116 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.094408 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.094597 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:02:55.098512 8711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:02:55.105168 8711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:55.136932 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.115950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55966 (local address 127.2.180.195:41779)
0808 15:02:55.129752 (+ 13802us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.129758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.129783 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:02:55.129848 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.129853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.129991 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.130146 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:55.130793 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.132325 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.134264 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.135555 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.135678 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.135930 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.136435 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.136607 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13669,"thread_start_us":13484,"threads_started":1}
I20260808 15:02:55.137005 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.115916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:41779 (local address 127.0.0.1:55966)
0808 15:02:55.127325 (+ 11409us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:55.127367 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:55.127534 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:55.130155 (+ 2621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:55.130159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:55.130190 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:55.130562 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.130577 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.132454 (+ 1877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.132459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.134114 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.134126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.135965 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.135972 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.136163 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.136259 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 15:02:55.136376 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 15:02:55.136538 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10638,"mutex_wait_us":72,"thread_start_us":11261,"threads_started":1}
I20260808 15:02:55.137033 9430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.128073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:37707 (local address 127.0.0.1:45466)
0808 15:02:55.128596 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:55.128613 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:55.128816 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:55.129713 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:55.129719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:55.129749 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:55.130099 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.130110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.131993 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.131998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.134408 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.134423 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.136200 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.136207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.136378 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.136410 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:02:55.136495 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 15:02:55.136617 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":10,"thread_start_us":339,"threads_started":1}
I20260808 15:02:55.137591 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.127412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45466 (local address 127.2.180.194:37707)
0808 15:02:55.128025 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.128035 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.128713 (+ 678us) server_negotiation.cc:407] Connection header received
0808 15:02:55.129003 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.129011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.129243 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.129406 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:55.130273 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.131835 (+ 1562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.134563 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.135992 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.136113 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.136288 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.137129 (+ 841us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.137293 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":304,"threads_started":1}
I20260808 15:02:55.138032 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.116585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35068 (local address 127.2.180.193:46235)
0808 15:02:55.129791 (+ 13206us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.129800 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.129825 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:02:55.129909 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.129916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.130120 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.130297 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:55.130999 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.132397 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.134873 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.136205 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.136333 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.136533 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.137590 (+ 1057us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.137740 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13039,"thread_start_us":13886,"threads_started":1}
I20260808 15:02:55.137472 9420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.116050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46235 (local address 127.0.0.1:35068)
0808 15:02:55.127441 (+ 11391us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:55.127455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:55.127546 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:55.130442 (+ 2896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:55.130446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:55.130471 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:55.130866 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.130876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.132523 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.132532 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.134695 (+ 2163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.134708 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.136686 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.136693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.136853 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.136891 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 15:02:55.136990 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 15:02:55.137105 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11275,"mutex_wait_us":22,"thread_start_us":10366,"threads_started":1}
I20260808 15:02:55.141215 8897 tablet_service.cc:1467] Processing CreateTablet for tablet f3276b4311dc48aa8c66b01e85e16881 (DEFAULT_TABLE table=test-table [id=28ff41b3956d4038a1cc754055bc0e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:55.141247 9302 tablet_service.cc:1467] Processing CreateTablet for tablet f3276b4311dc48aa8c66b01e85e16881 (DEFAULT_TABLE table=test-table [id=28ff41b3956d4038a1cc754055bc0e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:55.141661 9100 tablet_service.cc:1467] Processing CreateTablet for tablet f3276b4311dc48aa8c66b01e85e16881 (DEFAULT_TABLE table=test-table [id=28ff41b3956d4038a1cc754055bc0e0c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:55.142409 8897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3276b4311dc48aa8c66b01e85e16881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:55.142612 9302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3276b4311dc48aa8c66b01e85e16881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:55.142879 9100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3276b4311dc48aa8c66b01e85e16881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:55.146245 9001 heartbeater.cc:503] Master 127.2.180.254:45345 was elected leader, sending a full tablet report...
I20260808 15:02:55.153537 9432 tablet_bootstrap.cc:492] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: Bootstrap starting.
I20260808 15:02:55.156425 9433 tablet_bootstrap.cc:492] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: Bootstrap starting.
I20260808 15:02:55.157471 9432 tablet_bootstrap.cc:654] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:55.158722 9432 log.cc:824] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:55.159786 9433 tablet_bootstrap.cc:654] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:55.161026 9433 log.cc:824] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:55.161092 9434 tablet_bootstrap.cc:492] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Bootstrap starting.
I20260808 15:02:55.162484 9432 tablet_bootstrap.cc:492] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: No bootstrap required, opened a new log
I20260808 15:02:55.162783 9432 ts_tablet_manager.cc:1397] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 15:02:55.164006 9434 tablet_bootstrap.cc:654] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:55.164129 9433 tablet_bootstrap.cc:492] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: No bootstrap required, opened a new log
I20260808 15:02:55.164403 9433 ts_tablet_manager.cc:1397] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260808 15:02:55.165019 9434 log.cc:824] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:55.167397 9434 tablet_bootstrap.cc:492] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: No bootstrap required, opened a new log
I20260808 15:02:55.167615 9434 ts_tablet_manager.cc:1397] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260808 15:02:55.172757 9432 raft_consensus.cc:348] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.173169 9432 raft_consensus.cc:374] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:55.173275 9432 raft_consensus.cc:729] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff82246d19de49359140987a4c2405ed, State: Initialized, Role: FOLLOWER
I20260808 15:02:55.173856 9432 consensus_queue.cc:260] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.174883 9434 raft_consensus.cc:348] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.175196 9434 raft_consensus.cc:374] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:55.175288 9434 raft_consensus.cc:729] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bbff257a21445abbe19352ae9b29c60, State: Initialized, Role: FOLLOWER
I20260808 15:02:55.175037 9433 raft_consensus.cc:348] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.175428 9433 raft_consensus.cc:374] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:55.175535 9433 raft_consensus.cc:729] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b92e95b1e53f4def9095d8408deb733c, State: Initialized, Role: FOLLOWER
I20260808 15:02:55.175871 9434 consensus_queue.cc:260] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.176077 9433 consensus_queue.cc:260] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.176808 9432 ts_tablet_manager.cc:1428] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 15:02:55.177763 9199 heartbeater.cc:503] Master 127.2.180.254:45345 was elected leader, sending a full tablet report...
I20260808 15:02:55.178146 9400 heartbeater.cc:503] Master 127.2.180.254:45345 was elected leader, sending a full tablet report...
I20260808 15:02:55.178336 9433 ts_tablet_manager.cc:1428] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: Time spent starting tablet: real 0.014s user 0.012s sys 0.001s
I20260808 15:02:55.179885 9434 ts_tablet_manager.cc:1428] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Time spent starting tablet: real 0.012s user 0.005s sys 0.007s
I20260808 15:02:55.239223 9439 raft_consensus.cc:482] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:55.239464 9439 raft_consensus.cc:504] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.240993 9439 leader_election.cc:288] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ff82246d19de49359140987a4c2405ed (127.2.180.193:46235), b92e95b1e53f4def9095d8408deb733c (127.2.180.194:37707)
I20260808 15:02:55.247154 9403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.241306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46235 (local address 127.2.180.195:60675)
0808 15:02:55.241509 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:55.241525 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:55.241633 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:55.242097 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:55.242106 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:55.242132 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:55.242334 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.242346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.243554 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.243557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.245345 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.245360 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.246578 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.246585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.246714 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.246739 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 15:02:55.246838 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 15:02:55.246927 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 15:02:55.247218 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.241404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:60675 (local address 127.2.180.193:46235)
0808 15:02:55.241614 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.241620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.241640 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:02:55.241798 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.241801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.241857 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.241922 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:55.242473 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.243421 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.245489 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.246444 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.246603 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.246685 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.246865 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.246988 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:02:55.248023 9442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.241644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:37707 (local address 127.2.180.195:49897)
0808 15:02:55.242303 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:55.242318 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:55.242430 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:55.242723 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:55.242726 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:55.242744 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:02:55.242985 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.242992 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.244121 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.244126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.245864 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:55.245875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.247311 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.247317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:55.247461 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.247491 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 15:02:55.247579 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 15:02:55.247709 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":358,"threads_started":1}
I20260808 15:02:55.248360 8935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3276b4311dc48aa8c66b01e85e16881" candidate_uuid: "0bbff257a21445abbe19352ae9b29c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff82246d19de49359140987a4c2405ed" is_pre_election: true
I20260808 15:02:55.248656 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.241932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:49897 (local address 127.2.180.194:37707)
0808 15:02:55.242118 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.242123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.242395 (+ 272us) server_negotiation.cc:407] Connection header received
0808 15:02:55.242548 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.242553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.242624 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.242719 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:02:55.243123 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.243984 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.246003 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.247117 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.247279 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.247345 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.247625 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.247724 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:02:55.248847 8935 raft_consensus.cc:2454] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bbff257a21445abbe19352ae9b29c60 in term 0.
I20260808 15:02:55.249480 9233 leader_election.cc:302] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bbff257a21445abbe19352ae9b29c60, ff82246d19de49359140987a4c2405ed; no voters:
I20260808 15:02:55.249501 9133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3276b4311dc48aa8c66b01e85e16881" candidate_uuid: "0bbff257a21445abbe19352ae9b29c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92e95b1e53f4def9095d8408deb733c" is_pre_election: true
I20260808 15:02:55.249802 9439 raft_consensus.cc:2781] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:55.249936 9439 raft_consensus.cc:482] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:55.249944 9133 raft_consensus.cc:2454] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bbff257a21445abbe19352ae9b29c60 in term 0.
I20260808 15:02:55.250052 9439 raft_consensus.cc:3037] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:55.252599 9439 raft_consensus.cc:504] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.253085 9439 leader_election.cc:288] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [CANDIDATE]: Term 1 election: Requested vote from peers ff82246d19de49359140987a4c2405ed (127.2.180.193:46235), b92e95b1e53f4def9095d8408deb733c (127.2.180.194:37707)
I20260808 15:02:55.253515 8935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3276b4311dc48aa8c66b01e85e16881" candidate_uuid: "0bbff257a21445abbe19352ae9b29c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff82246d19de49359140987a4c2405ed"
I20260808 15:02:55.253583 9133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3276b4311dc48aa8c66b01e85e16881" candidate_uuid: "0bbff257a21445abbe19352ae9b29c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b92e95b1e53f4def9095d8408deb733c"
I20260808 15:02:55.253685 8935 raft_consensus.cc:3037] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:55.253753 9133 raft_consensus.cc:3037] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:55.255743 8935 raft_consensus.cc:2454] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bbff257a21445abbe19352ae9b29c60 in term 1.
I20260808 15:02:55.255877 9133 raft_consensus.cc:2454] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bbff257a21445abbe19352ae9b29c60 in term 1.
I20260808 15:02:55.256148 9233 leader_election.cc:302] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bbff257a21445abbe19352ae9b29c60, ff82246d19de49359140987a4c2405ed; no voters:
I20260808 15:02:55.256456 9439 raft_consensus.cc:2781] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:55.256870 9439 raft_consensus.cc:686] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 1 LEADER]: Becoming Leader. State: Replica: 0bbff257a21445abbe19352ae9b29c60, State: Running, Role: LEADER
I20260808 15:02:55.257303 9439 consensus_queue.cc:237] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } }
I20260808 15:02:55.261694 8710 catalog_manager.cc:5495] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bbff257a21445abbe19352ae9b29c60 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "0bbff257a21445abbe19352ae9b29c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bbff257a21445abbe19352ae9b29c60" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 } health_report { overall_health: UNKNOWN } } }
I20260808 15:02:55.292500 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.287436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55972 (local address 127.2.180.195:41779)
0808 15:02:55.287620 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.287627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.287710 (+ 83us) server_negotiation.cc:407] Connection header received
0808 15:02:55.287929 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.287933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.288014 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.288132 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:02:55.288502 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.289193 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.290513 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.290893 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.290998 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.291445 (+ 447us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:02:55.291948 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:02:55.292072 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.292194 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.292273 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:02:55.299749 9133 raft_consensus.cc:1261] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 1 FOLLOWER]: Refusing update from remote peer 0bbff257a21445abbe19352ae9b29c60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:55.299796 8935 raft_consensus.cc:1261] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 1 FOLLOWER]: Refusing update from remote peer 0bbff257a21445abbe19352ae9b29c60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:55.300606 9443 consensus_queue.cc:1035] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b92e95b1e53f4def9095d8408deb733c" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37707 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:55.300951 9439 consensus_queue.cc:1035] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff82246d19de49359140987a4c2405ed" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:02:55.305842 9401 tablet.cc:2369] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:02:55.310695 9449 mvcc.cc:205] Tried to move back new op lower bound from 7316280833219391488 to 7316280833057394688. Current Snapshot: MvccSnapshot[applied={T|T < 7316280833219391488}]
I20260808 15:02:55.312242 9452 mvcc.cc:205] Tried to move back new op lower bound from 7316280833219391488 to 7316280833057394688. Current Snapshot: MvccSnapshot[applied={T|T < 7316280833219391488}]
I20260808 15:02:55.324532 8712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:55.324734 8712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:55.327975 8712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 77241f5ece824fabab3054941c8eb9d5: Sending DeleteTablet for 3 replicas of tablet f3276b4311dc48aa8c66b01e85e16881
I20260808 15:02:55.328912 9302 tablet_service.cc:1514] Processing DeleteTablet for tablet f3276b4311dc48aa8c66b01e85e16881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:55 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:55966
I20260808 15:02:55.329066 8897 tablet_service.cc:1514] Processing DeleteTablet for tablet f3276b4311dc48aa8c66b01e85e16881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:55 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:35068
I20260808 15:02:55.329233 9100 tablet_service.cc:1514] Processing DeleteTablet for tablet f3276b4311dc48aa8c66b01e85e16881 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:55 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:45466
I20260808 15:02:55.332120 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 8805
I20260808 15:02:55.340114 9465 tablet_replica.cc:330] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: stopping tablet replica
I20260808 15:02:55.340356 9464 tablet_replica.cc:330] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: stopping tablet replica
I20260808 15:02:55.340641 9465 raft_consensus.cc:2229] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:55.340749 9464 raft_consensus.cc:2229] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:02:55.341179 9465 raft_consensus.cc:2258] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:55.341471 9464 raft_consensus.cc:2258] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:55.341971 9466 tablet_replica.cc:330] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: stopping tablet replica
I20260808 15:02:55.342600 9466 raft_consensus.cc:2229] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:55.344058 9465 ts_tablet_manager.cc:1905] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:55.344067 9464 ts_tablet_manager.cc:1905] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:55.344561 9466 raft_consensus.cc:2258] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:55.348459 9466 ts_tablet_manager.cc:1905] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:55.352464 9464 ts_tablet_manager.cc:1918] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:02:55.352686 9464 log.cc:1196] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/f3276b4311dc48aa8c66b01e85e16881
I20260808 15:02:55.352864 9465 ts_tablet_manager.cc:1918] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:02:55.353296 9464 ts_tablet_manager.cc:1939] T f3276b4311dc48aa8c66b01e85e16881 P 0bbff257a21445abbe19352ae9b29c60: Deleting consensus metadata
I20260808 15:02:55.354895 9466 ts_tablet_manager.cc:1918] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:02:55.355007 8693 catalog_manager.cc:4841] TS 0bbff257a21445abbe19352ae9b29c60 (127.2.180.195:41779): tablet f3276b4311dc48aa8c66b01e85e16881 (table test-table [id=28ff41b3956d4038a1cc754055bc0e0c]) successfully deleted
I20260808 15:02:55.355096 9466 log.cc:1196] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/f3276b4311dc48aa8c66b01e85e16881
I20260808 15:02:55.355602 9466 ts_tablet_manager.cc:1939] T f3276b4311dc48aa8c66b01e85e16881 P b92e95b1e53f4def9095d8408deb733c: Deleting consensus metadata
I20260808 15:02:55.355971 9465 log.cc:1196] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/f3276b4311dc48aa8c66b01e85e16881
Aug 08 15:02:55 dist-test-slave-jlzn krb5kdc[8653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:02:55.356544 9465 ts_tablet_manager.cc:1939] T f3276b4311dc48aa8c66b01e85e16881 P ff82246d19de49359140987a4c2405ed: Deleting consensus metadata
I20260808 15:02:55.357255 8696 catalog_manager.cc:4841] TS b92e95b1e53f4def9095d8408deb733c (127.2.180.194:37707): tablet f3276b4311dc48aa8c66b01e85e16881 (table test-table [id=28ff41b3956d4038a1cc754055bc0e0c]) successfully deleted
I20260808 15:02:55.358680 8696 catalog_manager.cc:4841] TS ff82246d19de49359140987a4c2405ed (127.2.180.193:46235): tablet f3276b4311dc48aa8c66b01e85e16881 (table test-table [id=28ff41b3956d4038a1cc754055bc0e0c]) successfully deleted
I20260808 15:02:55.362596 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.333418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35072 (local address 127.2.180.193:46235)
0808 15:02:55.333577 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.333583 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.342215 (+ 8632us) server_negotiation.cc:407] Connection header received
0808 15:02:55.342487 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.342493 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.342580 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.342662 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:55.344406 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.345436 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.347199 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.347771 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.347902 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.357241 (+ 9339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:55.357252 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:55.357264 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:55.357293 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:55.360167 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.360779 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.360785 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.360787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.360918 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.361259 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.361263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.361265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.361627 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:55.361900 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.362192 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.362342 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:02:55.363159 8992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:55.449301 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 8805
I20260808 15:02:55.470911 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 9011
Aug 08 15:02:55 dist-test-slave-jlzn krb5kdc[8653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:02:55.486680 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.471740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45482 (local address 127.2.180.194:37707)
0808 15:02:55.471981 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.471987 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.472009 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:02:55.472077 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.472083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.472163 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.472283 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:55.473409 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.474573 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.475714 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.476382 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.476511 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.480415 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:55.480435 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:55.480455 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:55.480502 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:55.484065 (+ 3563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.484656 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.484673 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.484677 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.484835 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.485140 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.485145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.485148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.485607 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:55.485928 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.486080 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.486307 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 15:02:55.487442 9192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:55.588887 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 9011
I20260808 15:02:55.609261 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 9205
Aug 08 15:02:55 dist-test-slave-jlzn krb5kdc[8653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201373, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:02:55.624656 9419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:55.610037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55974 (local address 127.2.180.195:41779)
0808 15:02:55.610228 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:55.610234 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:55.610320 (+ 86us) server_negotiation.cc:407] Connection header received
0808 15:02:55.610505 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:55.610510 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:55.610584 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:55.610673 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:55.611984 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.612844 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.614045 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:55.614502 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:55.614616 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:55.618828 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:55.618840 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:55.618856 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:02:55.618894 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:55.621830 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.622436 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.622444 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.622446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.622620 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:02:55.622987 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:02:55.622991 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:02:55.622993 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:02:55.623370 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:55.623709 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:55.623991 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 15:02:55.624222 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:02:55.625409 9394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:55.744557 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 9205
I20260808 15:02:55.768816 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 8675
I20260808 15:02:55.770017 8763 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:55.875936 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 8675
2026-08-08T15:02:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4092 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T15:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:55Z Disabled control of system clock
I20260808 15:02:55.946177 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:36947
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:36947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 15:02:56.254577 9481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:56.254853 9481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:56.254940 9481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:56.261776 9481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:56.261871 9481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:56.261919 9481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:56.261971 9481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:56.262014 9481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:02:56.270573 9481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:36947
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:36947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:56.271713 9481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:56.273373 9481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:56.284518 9489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:56.284591 9487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:56.284776 9486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:56.285590 9481 server_base.cc:1034] running on GCE node
I20260808 15:02:56.286266 9481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:56.287462 9481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:56.288645 9481 hybrid_clock.cc:648] HybridClock initialized: now 1786201376288594 us; error 40 us; skew 500 ppm
I20260808 15:02:56.288949 9481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:02:56.296036 9481 webserver.cc:460] Webserver started at http://127.2.180.254:41793/ using document root <none> and password file <none>
I20260808 15:02:56.296650 9481 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:56.296742 9481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:56.297041 9481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:56.299719 9481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "3e52614cf66d417783928581f19678c1"
format_stamp: "Formatted at 2026-08-08 15:02:56 on dist-test-slave-jlzn"
server_key: "435082ba6d19418b4fa7b3749d467bd3"
server_key_iv: "90dd06aaed9d46221bad40c2593e75d9"
server_key_version: "encryptionkey@0"
I20260808 15:02:56.300523 9481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "3e52614cf66d417783928581f19678c1"
format_stamp: "Formatted at 2026-08-08 15:02:56 on dist-test-slave-jlzn"
server_key: "435082ba6d19418b4fa7b3749d467bd3"
server_key_iv: "90dd06aaed9d46221bad40c2593e75d9"
server_key_version: "encryptionkey@0"
I20260808 15:02:56.305159 9481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:02:56.308236 9496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:56.309394 9481 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:02:56.309542 9481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "3e52614cf66d417783928581f19678c1"
format_stamp: "Formatted at 2026-08-08 15:02:56 on dist-test-slave-jlzn"
server_key: "435082ba6d19418b4fa7b3749d467bd3"
server_key_iv: "90dd06aaed9d46221bad40c2593e75d9"
server_key_version: "encryptionkey@0"
I20260808 15:02:56.309734 9481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:56.315068 9481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:56.315750 9481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:56.316064 9481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:56.332185 9481 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:36947
I20260808 15:02:56.332221 9572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:36947 every 8 connection(s)
I20260808 15:02:56.333312 9481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:56.336750 9573 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:56.341053 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 9481
I20260808 15:02:56.341241 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:56.342074 2771 external_mini_cluster.cc:1428] Setting key 697aa89047336ba1658d995eb76c51f9
I20260808 15:02:56.347615 9573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1: Bootstrap starting.
I20260808 15:02:56.349750 9578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:56.344019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44696 (local address 127.2.180.254:36947)
0808 15:02:56.344940 (+ 921us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:56.344952 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:56.344992 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:02:56.346598 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:56.346619 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:56.346960 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:56.347256 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:56.348315 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:56.348321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:56.348333 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:56.348346 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:56.348402 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:56.348417 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:56.348569 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:56.348698 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 15:02:56.348899 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":337,"threads_started":1}
I20260808 15:02:56.351020 9573 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:56.352082 9573 log.cc:824] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:56.354477 9573 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1: No bootstrap required, opened a new log
I20260808 15:02:56.361388 9573 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e52614cf66d417783928581f19678c1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 36947 } }
I20260808 15:02:56.361689 9573 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:56.361783 9573 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e52614cf66d417783928581f19678c1, State: Initialized, Role: FOLLOWER
I20260808 15:02:56.362329 9573 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e52614cf66d417783928581f19678c1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 36947 } }
I20260808 15:02:56.362543 9573 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:56.362668 9573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:56.362815 9573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:56.365005 9573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e52614cf66d417783928581f19678c1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 36947 } }
I20260808 15:02:56.365449 9573 leader_election.cc:302] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e52614cf66d417783928581f19678c1; no voters:
I20260808 15:02:56.365942 9573 leader_election.cc:288] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:56.366047 9580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:56.366680 9580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [term 1 LEADER]: Becoming Leader. State: Replica: 3e52614cf66d417783928581f19678c1, State: Running, Role: LEADER
I20260808 15:02:56.367260 9580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e52614cf66d417783928581f19678c1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 36947 } }
I20260808 15:02:56.367895 9573 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:56.369853 9582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e52614cf66d417783928581f19678c1. Latest consensus state: current_term: 1 leader_uuid: "3e52614cf66d417783928581f19678c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e52614cf66d417783928581f19678c1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 36947 } } }
I20260808 15:02:56.370232 9582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:56.370779 9581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e52614cf66d417783928581f19678c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e52614cf66d417783928581f19678c1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 36947 } } }
I20260808 15:02:56.371173 9581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:56.372308 9592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:56.376974 9592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:56.384763 9592 catalog_manager.cc:1347] Generated new cluster ID: 33510d4f10af4cd6b5d5ff408dd403d3
I20260808 15:02:56.384868 9592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:56.398154 9592 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:56.398558 9592 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:56.423494 9592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1: Generated new TSK 0
I20260808 15:02:56.424276 9592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:02:56.433460 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:36947
--builtin_ntp_servers=127.2.180.212:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 15:02:56.761508 9604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:56.761770 9604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:56.761902 9604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:56.768823 9604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:56.768911 9604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:56.769057 9604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:02:56.777685 9604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:36947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:56.778805 9604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:56.780516 9604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:56.792706 9613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:56.792735 9615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:56.792838 9617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:56.793910 9604 server_base.cc:1034] running on GCE node
I20260808 15:02:56.794422 9604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:56.795230 9604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:56.796420 9604 hybrid_clock.cc:648] HybridClock initialized: now 1786201376796360 us; error 50 us; skew 500 ppm
I20260808 15:02:56.796718 9604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:02:56.798662 9604 webserver.cc:460] Webserver started at http://127.2.180.193:36131/ using document root <none> and password file <none>
I20260808 15:02:56.799286 9604 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:56.799394 9604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:56.799695 9604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:56.802386 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd"
format_stamp: "Formatted at 2026-08-08 15:02:56 on dist-test-slave-jlzn"
server_key: "db088fcef5ac31debc03499d0d15e38b"
server_key_iv: "f98171edf230d62c12bd2322daa19f0b"
server_key_version: "encryptionkey@0"
I20260808 15:02:56.803133 9604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd"
format_stamp: "Formatted at 2026-08-08 15:02:56 on dist-test-slave-jlzn"
server_key: "db088fcef5ac31debc03499d0d15e38b"
server_key_iv: "f98171edf230d62c12bd2322daa19f0b"
server_key_version: "encryptionkey@0"
I20260808 15:02:56.807907 9604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 15:02:56.811026 9626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:56.812269 9604 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:02:56.812419 9604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd"
format_stamp: "Formatted at 2026-08-08 15:02:56 on dist-test-slave-jlzn"
server_key: "db088fcef5ac31debc03499d0d15e38b"
server_key_iv: "f98171edf230d62c12bd2322daa19f0b"
server_key_version: "encryptionkey@0"
I20260808 15:02:56.812664 9604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:56.817865 9604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:56.818554 9604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:56.818795 9604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:56.819710 9604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:56.820986 9604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:56.821089 9604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:56.821197 9604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:56.821264 9604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:56.850478 9604 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:39849
I20260808 15:02:56.850525 9802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:39849 every 8 connection(s)
I20260808 15:02:56.851686 9604 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:02:56.855325 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 9604
I20260808 15:02:56.855507 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:02:56.856199 2771 external_mini_cluster.cc:1428] Setting key f122a5e4df861bf4962963b7273fc9a1
I20260808 15:02:56.859100 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:36947
--builtin_ntp_servers=127.2.180.212:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 15:02:56.859788 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:56.854017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:43131 (local address 127.2.180.254:36947)
0808 15:02:56.854438 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:56.854442 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:56.855618 (+ 1176us) server_negotiation.cc:407] Connection header received
0808 15:02:56.857257 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:56.857261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:56.857310 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:56.857391 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:56.858723 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:56.858729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:56.858733 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:56.858755 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:56.858806 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:56.858820 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:56.858936 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:56.859247 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 15:02:56.859342 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":212,"threads_started":1}
I20260808 15:02:56.860503 9807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:56.854502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:36947 (local address 127.2.180.193:43131)
0808 15:02:56.855338 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:56.855379 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:56.856970 (+ 1591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:56.857638 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:56.857648 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:56.857759 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:56.858385 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:02:56.858388 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:56.858438 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:02:56.858535 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:56.858949 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:56.858961 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:56.858969 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:02:56.859303 (+ 334us) client_negotiation.cc:240] Negotiation successful
0808 15:02:56.859539 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":256,"threads_started":1}
I20260808 15:02:56.861829 9803 heartbeater.cc:347] Connected to a master server at 127.2.180.254:36947
I20260808 15:02:56.862051 9803 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:56.862548 9803 heartbeater.cc:511] Master 127.2.180.254:36947 requested a full tablet report, sending...
I20260808 15:02:56.864703 9519 ts_manager.cc:194] Registered new tserver with Master: c6d3581ca4bb42fa80e8f6d43a356bfd (127.2.180.193:39849)
W20260808 15:02:57.173308 9808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:57.173591 9808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:57.173718 9808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:57.180836 9808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:57.180931 9808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:57.181087 9808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:02:57.189776 9808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:36947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:57.190933 9808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:57.192651 9808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:57.205026 9815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:57.204963 9816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:57.205000 9819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:57.205794 9808 server_base.cc:1034] running on GCE node
I20260808 15:02:57.206314 9808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:57.207104 9808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:57.208290 9808 hybrid_clock.cc:648] HybridClock initialized: now 1786201377208245 us; error 37 us; skew 500 ppm
I20260808 15:02:57.208572 9808 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:02:57.210570 9808 webserver.cc:460] Webserver started at http://127.2.180.194:34141/ using document root <none> and password file <none>
I20260808 15:02:57.211197 9808 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:57.211293 9808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:57.211608 9808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:57.214349 9808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31"
format_stamp: "Formatted at 2026-08-08 15:02:57 on dist-test-slave-jlzn"
server_key: "9525d7ebd4c5ee26aa12d8518faa95df"
server_key_iv: "4db1d578f62cf18c2e521ef699f65f02"
server_key_version: "encryptionkey@0"
I20260808 15:02:57.215128 9808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31"
format_stamp: "Formatted at 2026-08-08 15:02:57 on dist-test-slave-jlzn"
server_key: "9525d7ebd4c5ee26aa12d8518faa95df"
server_key_iv: "4db1d578f62cf18c2e521ef699f65f02"
server_key_version: "encryptionkey@0"
I20260808 15:02:57.219838 9808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 15:02:57.222950 9827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:57.224179 9808 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260808 15:02:57.224324 9808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31"
format_stamp: "Formatted at 2026-08-08 15:02:57 on dist-test-slave-jlzn"
server_key: "9525d7ebd4c5ee26aa12d8518faa95df"
server_key_iv: "4db1d578f62cf18c2e521ef699f65f02"
server_key_version: "encryptionkey@0"
I20260808 15:02:57.224529 9808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:57.229676 9808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:57.230373 9808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:57.230607 9808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:57.231433 9808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:57.232754 9808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:57.232842 9808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:57.232949 9808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:57.233019 9808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:57.261896 9808 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:38945
I20260808 15:02:57.261946 9995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:38945 every 8 connection(s)
I20260808 15:02:57.263094 9808 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:02:57.270277 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 9808
I20260808 15:02:57.270418 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:02:57.270591 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.265379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:45597 (local address 127.2.180.254:36947)
0808 15:02:57.265547 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.265551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.266819 (+ 1268us) server_negotiation.cc:407] Connection header received
0808 15:02:57.268512 (+ 1693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.268518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.268579 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.268669 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.269866 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.269872 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.269876 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.269892 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.269925 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.269936 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.270038 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.270219 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.270287 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:02:57.270974 2771 external_mini_cluster.cc:1428] Setting key bf0ffdc1feefc40c8038f27ba580bff5
I20260808 15:02:57.271407 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.265818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:36947 (local address 127.2.180.194:45597)
0808 15:02:57.266664 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:57.266705 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:57.268238 (+ 1533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:57.268903 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:57.268912 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:57.268994 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:57.269579 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:02:57.269581 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:57.269617 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:02:57.269700 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:57.270022 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:57.270032 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:57.270040 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:02:57.270285 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 15:02:57.270541 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":256,"threads_started":1}
I20260808 15:02:57.272754 9997 heartbeater.cc:347] Connected to a master server at 127.2.180.254:36947
I20260808 15:02:57.273020 9997 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:57.273537 9997 heartbeater.cc:511] Master 127.2.180.254:36947 requested a full tablet report, sending...
I20260808 15:02:57.273550 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:36947
--builtin_ntp_servers=127.2.180.212:39237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 15:02:57.274784 9519 ts_manager.cc:194] Registered new tserver with Master: 4b7fbc2806bb4ffe9d3b54b7f102be31 (127.2.180.194:38945)
W20260808 15:02:57.585732 10001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:57.586027 10001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:57.586151 10001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:57.593180 10001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:57.593267 10001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:57.593406 10001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:02:57.602011 10001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:36947
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:57.603173 10001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:57.604966 10001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:57.617256 10011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:57.617357 10009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:57.617357 10013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:57.618384 10001 server_base.cc:1034] running on GCE node
I20260808 15:02:57.618918 10001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:57.619751 10001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:57.620962 10001 hybrid_clock.cc:648] HybridClock initialized: now 1786201377620921 us; error 34 us; skew 500 ppm
I20260808 15:02:57.621258 10001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:02:57.623301 10001 webserver.cc:460] Webserver started at http://127.2.180.195:38547/ using document root <none> and password file <none>
I20260808 15:02:57.623962 10001 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:57.624066 10001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:57.624375 10001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:57.627081 10001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "158c9270450e47d291be12d3d7016e6b"
format_stamp: "Formatted at 2026-08-08 15:02:57 on dist-test-slave-jlzn"
server_key: "54e5e2620def036cda229ebb84efee7d"
server_key_iv: "9e1f851039e2126e08f2d2ce5134d650"
server_key_version: "encryptionkey@0"
I20260808 15:02:57.627907 10001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "158c9270450e47d291be12d3d7016e6b"
format_stamp: "Formatted at 2026-08-08 15:02:57 on dist-test-slave-jlzn"
server_key: "54e5e2620def036cda229ebb84efee7d"
server_key_iv: "9e1f851039e2126e08f2d2ce5134d650"
server_key_version: "encryptionkey@0"
I20260808 15:02:57.632704 10001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 15:02:57.635874 10024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:57.637006 10001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:02:57.637144 10001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "158c9270450e47d291be12d3d7016e6b"
format_stamp: "Formatted at 2026-08-08 15:02:57 on dist-test-slave-jlzn"
server_key: "54e5e2620def036cda229ebb84efee7d"
server_key_iv: "9e1f851039e2126e08f2d2ce5134d650"
server_key_version: "encryptionkey@0"
I20260808 15:02:57.637349 10001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:57.642813 10001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:57.643548 10001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:57.643778 10001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:57.644791 10001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:02:57.646232 10001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:02:57.646334 10001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:57.646440 10001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:02:57.646509 10001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:57.675052 10001 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:36671
I20260808 15:02:57.675103 10207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:36671 every 8 connection(s)
I20260808 15:02:57.676266 10001 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:02:57.681155 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 10001
I20260808 15:02:57.681295 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:02:57.681736 2771 external_mini_cluster.cc:1428] Setting key 7ecfc84827c52946f008b491aec5c457
I20260808 15:02:57.684018 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.678621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:53147 (local address 127.2.180.254:36947)
0808 15:02:57.678791 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.678796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.680124 (+ 1328us) server_negotiation.cc:407] Connection header received
0808 15:02:57.681871 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.681877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.681941 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.682031 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.683290 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.683296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.683301 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.683317 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.683349 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.683360 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.683470 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.683657 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.683721 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:02:57.684852 10211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.679097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:36947 (local address 127.2.180.195:53147)
0808 15:02:57.679964 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:57.680006 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:57.681606 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:57.682261 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:57.682268 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:57.682352 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:57.682985 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:02:57.682989 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:57.683033 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:02:57.683129 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:57.683446 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:57.683457 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:57.683464 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 15:02:57.683713 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 15:02:57.683970 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":253,"threads_started":1}
I20260808 15:02:57.685915 10209 heartbeater.cc:347] Connected to a master server at 127.2.180.254:36947
I20260808 15:02:57.686177 10209 heartbeater.cc:464] Registering TS with master...
I20260808 15:02:57.686702 10209 heartbeater.cc:511] Master 127.2.180.254:36947 requested a full tablet report, sending...
I20260808 15:02:57.688009 9519 ts_manager.cc:194] Registered new tserver with Master: 158c9270450e47d291be12d3d7016e6b (127.2.180.195:36671)
I20260808 15:02:57.696570 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:02:57.703012 9805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.700730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44710 (local address 127.2.180.254:36947)
0808 15:02:57.700864 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.700870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.701318 (+ 448us) server_negotiation.cc:407] Connection header received
0808 15:02:57.701463 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.701468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.701516 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.701601 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.702436 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.702441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.702446 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.702463 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.702492 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.702503 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.702612 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.702709 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.702766 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:02:57.706382 9519 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44710:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:02:57.712880 9519 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:02:57.728878 10226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.724030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.2.180.194:38945)
0808 15:02:57.724604 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.724612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.725333 (+ 721us) server_negotiation.cc:407] Connection header received
0808 15:02:57.725408 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.725415 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.725636 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.725789 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.727707 (+ 1918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.727716 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.727730 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.727758 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.727841 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.727867 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.728036 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.728395 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.728518 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":266,"threads_started":1}
I20260808 15:02:57.729206 10232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.725361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:36671 (local address 127.0.0.1:55216)
0808 15:02:57.725830 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:57.725845 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:57.725929 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:57.727067 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:57.727071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:57.727094 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:57.727690 (+ 596us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:02:57.727693 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:57.727709 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:02:57.727751 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:57.728423 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:57.728427 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:57.728432 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:02:57.728588 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 15:02:57.728667 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":779,"threads_started":1}
I20260808 15:02:57.729686 10225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.723885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:38945 (local address 127.0.0.1:58350)
0808 15:02:57.725003 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:57.725046 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:57.725208 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:57.726543 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:57.726550 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:57.726637 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:57.727387 (+ 750us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:02:57.727391 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:57.727431 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:02:57.727509 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:57.728144 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:57.728154 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:57.728161 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 15:02:57.729139 (+ 978us) client_negotiation.cc:240] Negotiation successful
0808 15:02:57.729198 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"thread_start_us":724,"threads_started":1}
I20260808 15:02:57.729897 10233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.725888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55216 (local address 127.2.180.195:36671)
0808 15:02:57.726620 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.726629 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.726663 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:02:57.726727 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.726735 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.726926 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.727062 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.728141 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.728148 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.728161 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.728179 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.728231 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.728252 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.729382 (+ 1130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.729444 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.729526 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":381,"threads_started":1}
I20260808 15:02:57.730887 10229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.724252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58388 (local address 127.2.180.193:39849)
0808 15:02:57.727629 (+ 3377us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.727643 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.727674 (+ 31us) server_negotiation.cc:407] Connection header received
0808 15:02:57.727743 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.727750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.727965 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.729311 (+ 1346us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.730128 (+ 817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.730135 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.730146 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.730175 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.730231 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.730251 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.730379 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.730508 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.730604 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2989,"thread_start_us":2588,"threads_started":1}
I20260808 15:02:57.730996 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.724637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58388)
0808 15:02:57.725252 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:57.725268 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:57.725363 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:57.728146 (+ 2783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:57.728162 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:57.728184 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:57.729018 (+ 834us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:02:57.729022 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:57.729040 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:02:57.729085 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:57.730367 (+ 1282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:57.730372 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:57.730377 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:02:57.730524 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:02:57.730583 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":330,"threads_started":1}
I20260808 15:02:57.733870 9695 tablet_service.cc:1467] Processing CreateTablet for tablet c964cb9fc1e0408389761554465f70b9 (DEFAULT_TABLE table=test-table [id=fb512dfbf6774fb6b32e7c2762917db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:57.733740 10107 tablet_service.cc:1467] Processing CreateTablet for tablet c964cb9fc1e0408389761554465f70b9 (DEFAULT_TABLE table=test-table [id=fb512dfbf6774fb6b32e7c2762917db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:57.733744 9898 tablet_service.cc:1467] Processing CreateTablet for tablet c964cb9fc1e0408389761554465f70b9 (DEFAULT_TABLE table=test-table [id=fb512dfbf6774fb6b32e7c2762917db9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:02:57.734962 9695 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c964cb9fc1e0408389761554465f70b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:57.735088 10107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c964cb9fc1e0408389761554465f70b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:57.735105 9898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c964cb9fc1e0408389761554465f70b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:57.745591 10237 tablet_bootstrap.cc:492] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd: Bootstrap starting.
I20260808 15:02:57.748831 10238 tablet_bootstrap.cc:492] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: Bootstrap starting.
I20260808 15:02:57.749226 10237 tablet_bootstrap.cc:654] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:57.750218 10239 tablet_bootstrap.cc:492] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: Bootstrap starting.
I20260808 15:02:57.750466 10237 log.cc:824] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:57.752072 10238 tablet_bootstrap.cc:654] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:57.753238 10238 log.cc:824] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:57.753314 10239 tablet_bootstrap.cc:654] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:57.753561 10237 tablet_bootstrap.cc:492] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd: No bootstrap required, opened a new log
I20260808 15:02:57.753870 10237 ts_tablet_manager.cc:1397] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 15:02:57.754448 10239 log.cc:824] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:57.757050 10238 tablet_bootstrap.cc:492] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: No bootstrap required, opened a new log
I20260808 15:02:57.757275 10238 ts_tablet_manager.cc:1397] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 15:02:57.757308 10239 tablet_bootstrap.cc:492] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: No bootstrap required, opened a new log
I20260808 15:02:57.757539 10239 ts_tablet_manager.cc:1397] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 15:02:57.763620 10237 raft_consensus.cc:348] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.764071 10237 raft_consensus.cc:374] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:57.764166 10237 raft_consensus.cc:729] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6d3581ca4bb42fa80e8f6d43a356bfd, State: Initialized, Role: FOLLOWER
I20260808 15:02:57.764123 10238 raft_consensus.cc:348] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.764645 10238 raft_consensus.cc:374] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:57.764741 10238 raft_consensus.cc:729] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 158c9270450e47d291be12d3d7016e6b, State: Initialized, Role: FOLLOWER
I20260808 15:02:57.764750 10237 consensus_queue.cc:260] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.765257 10238 consensus_queue.cc:260] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.765897 9803 heartbeater.cc:503] Master 127.2.180.254:36947 was elected leader, sending a full tablet report...
I20260808 15:02:57.766481 10237 ts_tablet_manager.cc:1428] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd: Time spent starting tablet: real 0.012s user 0.007s sys 0.004s
I20260808 15:02:57.768317 10239 raft_consensus.cc:348] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.768796 10239 raft_consensus.cc:374] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:57.768882 10239 raft_consensus.cc:729] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b7fbc2806bb4ffe9d3b54b7f102be31, State: Initialized, Role: FOLLOWER
I20260808 15:02:57.769613 10239 consensus_queue.cc:260] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.773761 10209 heartbeater.cc:503] Master 127.2.180.254:36947 was elected leader, sending a full tablet report...
I20260808 15:02:57.773723 10246 raft_consensus.cc:482] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:02:57.774012 10246 raft_consensus.cc:504] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.774770 10238 ts_tablet_manager.cc:1428] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: Time spent starting tablet: real 0.017s user 0.005s sys 0.012s
I20260808 15:02:57.775563 9997 heartbeater.cc:503] Master 127.2.180.254:36947 was elected leader, sending a full tablet report...
I20260808 15:02:57.775980 10246 leader_election.cc:288] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b7fbc2806bb4ffe9d3b54b7f102be31 (127.2.180.194:38945), c6d3581ca4bb42fa80e8f6d43a356bfd (127.2.180.193:39849)
I20260808 15:02:57.777172 10239 ts_tablet_manager.cc:1428] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: Time spent starting tablet: real 0.020s user 0.009s sys 0.007s
I20260808 15:02:57.779412 10211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.776092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:38945 (local address 127.2.180.195:51073)
0808 15:02:57.776737 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:57.776757 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:57.776848 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:57.777864 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:57.777869 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:57.777896 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:57.778549 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:02:57.778552 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:57.778570 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:02:57.778617 (+ 47us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:57.779010 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:57.779014 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:57.779019 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:02:57.779129 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 15:02:57.779180 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529}
I20260808 15:02:57.780263 10226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.776876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:51073 (local address 127.2.180.194:38945)
0808 15:02:57.777252 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.777258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.777275 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:02:57.777328 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.777332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.777383 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.777476 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.778774 (+ 1298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.778779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.778783 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.778802 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.778837 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.778851 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.778959 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.779968 (+ 1009us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.780035 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265}
I20260808 15:02:57.781279 9928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c964cb9fc1e0408389761554465f70b9" candidate_uuid: "158c9270450e47d291be12d3d7016e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" is_pre_election: true
I20260808 15:02:57.781550 10211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.776476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.2.180.195:53323)
0808 15:02:57.779607 (+ 3131us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:57.779624 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:57.779704 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:57.780102 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:57.780107 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:57.780130 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:02:57.780663 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:02:57.780666 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:02:57.780682 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:02:57.780725 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:02:57.781114 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:02:57.781118 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:02:57.781123 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:02:57.781215 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 15:02:57.781265 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3036,"threads_started":1}
I20260808 15:02:57.781713 9928 raft_consensus.cc:2454] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 158c9270450e47d291be12d3d7016e6b in term 0.
I20260808 15:02:57.781998 10229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.777037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:53323 (local address 127.2.180.193:39849)
0808 15:02:57.777219 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.777224 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.779780 (+ 2556us) server_negotiation.cc:407] Connection header received
0808 15:02:57.779897 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.779902 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.779959 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.780060 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.780879 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.780884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.780888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.780907 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.780941 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.780956 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.781059 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.781705 (+ 646us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.781763 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:02:57.782434 10027 leader_election.cc:302] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158c9270450e47d291be12d3d7016e6b, 4b7fbc2806bb4ffe9d3b54b7f102be31; no voters:
I20260808 15:02:57.782814 10248 raft_consensus.cc:2781] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:02:57.782938 10248 raft_consensus.cc:482] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:02:57.783111 10248 raft_consensus.cc:3037] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:57.784891 9726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c964cb9fc1e0408389761554465f70b9" candidate_uuid: "158c9270450e47d291be12d3d7016e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" is_pre_election: true
I20260808 15:02:57.785287 9726 raft_consensus.cc:2454] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 158c9270450e47d291be12d3d7016e6b in term 0.
I20260808 15:02:57.786057 10248 raft_consensus.cc:504] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.786731 10248 leader_election.cc:288] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [CANDIDATE]: Term 1 election: Requested vote from peers 4b7fbc2806bb4ffe9d3b54b7f102be31 (127.2.180.194:38945), c6d3581ca4bb42fa80e8f6d43a356bfd (127.2.180.193:39849)
I20260808 15:02:57.787072 9928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c964cb9fc1e0408389761554465f70b9" candidate_uuid: "158c9270450e47d291be12d3d7016e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31"
I20260808 15:02:57.787248 9928 raft_consensus.cc:3037] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:57.787223 9726 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c964cb9fc1e0408389761554465f70b9" candidate_uuid: "158c9270450e47d291be12d3d7016e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd"
I20260808 15:02:57.787386 9726 raft_consensus.cc:3037] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:57.789502 9928 raft_consensus.cc:2454] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 158c9270450e47d291be12d3d7016e6b in term 1.
I20260808 15:02:57.789809 9726 raft_consensus.cc:2454] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 158c9270450e47d291be12d3d7016e6b in term 1.
I20260808 15:02:57.789844 10027 leader_election.cc:302] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 158c9270450e47d291be12d3d7016e6b, 4b7fbc2806bb4ffe9d3b54b7f102be31; no voters:
I20260808 15:02:57.790102 10248 raft_consensus.cc:2781] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:57.790267 10248 raft_consensus.cc:686] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 1 LEADER]: Becoming Leader. State: Replica: 158c9270450e47d291be12d3d7016e6b, State: Running, Role: LEADER
I20260808 15:02:57.790661 10248 consensus_queue.cc:237] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } }
I20260808 15:02:57.794544 9519 catalog_manager.cc:5495] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 158c9270450e47d291be12d3d7016e6b (127.2.180.195). New cstate: current_term: 1 leader_uuid: "158c9270450e47d291be12d3d7016e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "158c9270450e47d291be12d3d7016e6b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 36671 } health_report { overall_health: HEALTHY } } }
I20260808 15:02:57.809641 10233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.808107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.2.180.195:36671)
0808 15:02:57.808246 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.808251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.808265 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:02:57.808336 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.808340 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.808408 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.808485 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.809243 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.809248 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.809251 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.809265 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.809293 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.809304 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.809397 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.809432 (+ 35us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.809473 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:02:57.817333 9928 raft_consensus.cc:1261] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 1 FOLLOWER]: Refusing update from remote peer 158c9270450e47d291be12d3d7016e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:57.817350 9726 raft_consensus.cc:1261] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd [term 1 FOLLOWER]: Refusing update from remote peer 158c9270450e47d291be12d3d7016e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:02:57.818184 10246 consensus_queue.cc:1035] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b7fbc2806bb4ffe9d3b54b7f102be31" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 38945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:57.818476 10248 consensus_queue.cc:1035] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6d3581ca4bb42fa80e8f6d43a356bfd" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 39849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:02:57.829387 10258 mvcc.cc:205] Tried to move back new op lower bound from 7316280843530756096 to 7316280843433951232. Current Snapshot: MvccSnapshot[applied={T|T < 7316280843530756096}]
I20260808 15:02:57.841109 9518 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:57.841393 9518 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44710:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:02:57.845242 9518 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e52614cf66d417783928581f19678c1: Sending DeleteTablet for 3 replicas of tablet c964cb9fc1e0408389761554465f70b9
I20260808 15:02:57.846309 9695 tablet_service.cc:1514] Processing DeleteTablet for tablet c964cb9fc1e0408389761554465f70b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:57 UTC) from {username='slave'} at 127.0.0.1:58388
I20260808 15:02:57.846300 10107 tablet_service.cc:1514] Processing DeleteTablet for tablet c964cb9fc1e0408389761554465f70b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:57 UTC) from {username='slave'} at 127.0.0.1:55216
I20260808 15:02:57.846654 9898 tablet_service.cc:1514] Processing DeleteTablet for tablet c964cb9fc1e0408389761554465f70b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:02:57 UTC) from {username='slave'} at 127.0.0.1:58350
I20260808 15:02:57.848183 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 9604
I20260808 15:02:57.860221 10272 tablet_replica.cc:330] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: stopping tablet replica
I20260808 15:02:57.861606 10272 raft_consensus.cc:2229] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:02:57.862306 10229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:57.849485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58404 (local address 127.2.180.193:39849)
0808 15:02:57.849666 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:57.849673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:57.857793 (+ 8120us) server_negotiation.cc:407] Connection header received
0808 15:02:57.858620 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:57.858626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:57.858737 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:57.858812 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:57.860837 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:57.860843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:57.860848 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:57.860870 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:57.860911 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:57.860926 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:57.861024 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:57.861896 (+ 872us) server_negotiation.cc:299] Negotiation successful
0808 15:02:57.861967 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:02:57.863355 9796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:57.863701 10272 raft_consensus.cc:2258] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:57.864753 10271 tablet_replica.cc:330] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: stopping tablet replica
I20260808 15:02:57.866304 10272 ts_tablet_manager.cc:1905] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:57.869282 10271 raft_consensus.cc:2229] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:02:57.870023 10271 raft_consensus.cc:2258] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:02:57.876209 10271 ts_tablet_manager.cc:1905] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:02:57.879089 10272 ts_tablet_manager.cc:1918] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:02:57.879289 10272 log.cc:1196] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/c964cb9fc1e0408389761554465f70b9
I20260808 15:02:57.879887 10272 ts_tablet_manager.cc:1939] T c964cb9fc1e0408389761554465f70b9 P 4b7fbc2806bb4ffe9d3b54b7f102be31: Deleting consensus metadata
I20260808 15:02:57.881752 9497 catalog_manager.cc:4841] TS 4b7fbc2806bb4ffe9d3b54b7f102be31 (127.2.180.194:38945): tablet c964cb9fc1e0408389761554465f70b9 (table test-table [id=fb512dfbf6774fb6b32e7c2762917db9]) successfully deleted
I20260808 15:02:57.882925 10271 ts_tablet_manager.cc:1918] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:02:57.883078 10271 log.cc:1196] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/c964cb9fc1e0408389761554465f70b9
I20260808 15:02:57.883515 10271 ts_tablet_manager.cc:1939] T c964cb9fc1e0408389761554465f70b9 P 158c9270450e47d291be12d3d7016e6b: Deleting consensus metadata
I20260808 15:02:57.884838 9497 catalog_manager.cc:4841] TS 158c9270450e47d291be12d3d7016e6b (127.2.180.195:36671): tablet c964cb9fc1e0408389761554465f70b9 (table test-table [id=fb512dfbf6774fb6b32e7c2762917db9]) successfully deleted
I20260808 15:02:58.021076 10269 tablet_replica.cc:330] T c964cb9fc1e0408389761554465f70b9 P c6d3581ca4bb42fa80e8f6d43a356bfd: stopping tablet replica
I20260808 15:02:58.021430 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 9604
I20260808 15:02:58.046612 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 9808
W20260808 15:02:58.046583 9499 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.180.193:39849 (error 108)
I20260808 15:02:58.049131 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.048615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58406)
0808 15:02:58.048816 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:58.048992 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 15:02:58.049348 9499 catalog_manager.cc:4584] TS c6d3581ca4bb42fa80e8f6d43a356bfd (127.2.180.193:39849): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c964cb9fc1e0408389761554465f70b9: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
I20260808 15:02:58.050275 10226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.047461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58354 (local address 127.2.180.194:38945)
0808 15:02:58.047642 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:58.047648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:58.047669 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:02:58.047912 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:58.047918 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:58.048030 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:58.048149 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:58.049626 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:58.049632 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:58.049636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:58.049657 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:58.049699 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:58.049714 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:58.049836 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:58.049950 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 15:02:58.050012 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:02:58.050978 9987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:58.102718 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.102281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58408)
0808 15:02:58.102460 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:58.102539 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 15:02:58.103688 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.103347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58424)
0808 15:02:58.103493 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:58.103576 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 15:02:58.164242 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.163750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58426)
0808 15:02:58.163956 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:58.164081 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 15:02:58.165326 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.164984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58438)
0808 15:02:58.165132 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:58.165204 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 15:02:58.194695 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 9808
I20260808 15:02:58.223522 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 10001
I20260808 15:02:58.226518 10233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.224429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55234 (local address 127.2.180.195:36671)
0808 15:02:58.224649 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:58.224655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:58.224680 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:02:58.224837 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:58.224842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:58.224898 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:58.224983 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:58.225875 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:58.225881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:58.225885 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:58.225900 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:58.225932 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:58.225943 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:58.226057 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:58.226155 (+ 98us) server_negotiation.cc:299] Negotiation successful
0808 15:02:58.226216 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 15:02:58.227265 10201 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:58.246029 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.245630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58452)
0808 15:02:58.245808 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:58.245899 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 15:02:58.246984 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.246618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58462)
0808 15:02:58.246800 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:58.246868 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 15:02:58.343768 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 10001
I20260808 15:02:58.365911 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 9481
I20260808 15:02:58.367012 9567 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:58.469439 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.468950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:39849 (local address 127.0.0.1:58464)
0808 15:02:58.469143 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:58.469234 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:39849: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 15:02:58.469514 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 9481
2026-08-08T15:02:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2591 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 15:02:58.513808 2771 mini_oidc.cc:152] Starting JWKS server
I20260808 15:02:58.516713 2771 webserver.cc:460] Webserver started at https://127.0.0.1:46851/ using document root <none> and password file <none>
I20260808 15:02:58.517030 2771 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 15:02:58.517616 2771 webserver.cc:460] Webserver started at http://127.0.0.1:34075/ using document root <none> and password file <none>
2026-08-08T15:02:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:58Z Disabled control of system clock
I20260808 15:02:58.543713 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40541
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:40917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40541
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46851/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/testchainca.cert with env {}
W20260808 15:02:58.851969 10286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:58.852228 10286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:58.852319 10286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:58.859114 10286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:58.859206 10286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:58.859251 10286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:58.859306 10286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:58.859349 10286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 15:02:58.859408 10286 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 15:02:58.859447 10286 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46851/jwks.json
I20260808 15:02:58.870500 10286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46851/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:58.871650 10286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:58.872859 10286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:58.883630 10294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:58.883695 10296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:58.883726 10293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:58.884452 10286 server_base.cc:1034] running on GCE node
I20260808 15:02:58.885157 10286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:58.886427 10286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:58.887638 10286 hybrid_clock.cc:648] HybridClock initialized: now 1786201378887588 us; error 42 us; skew 500 ppm
I20260808 15:02:58.888008 10286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:02:58.890019 10286 webserver.cc:460] Webserver started at http://127.2.180.254:35171/ using document root <none> and password file <none>
I20260808 15:02:58.890619 10286 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:58.890731 10286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:58.891009 10286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:58.893746 10286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "6d719451453945caae01fb4542a4fd16"
format_stamp: "Formatted at 2026-08-08 15:02:58 on dist-test-slave-jlzn"
server_key: "613f14f6e6f0c5d03238a2f48bb27696"
server_key_iv: "cd39e2ba642da6a3084c6b0d23930bc5"
server_key_version: "encryptionkey@0"
I20260808 15:02:58.894459 10286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "6d719451453945caae01fb4542a4fd16"
format_stamp: "Formatted at 2026-08-08 15:02:58 on dist-test-slave-jlzn"
server_key: "613f14f6e6f0c5d03238a2f48bb27696"
server_key_iv: "cd39e2ba642da6a3084c6b0d23930bc5"
server_key_version: "encryptionkey@0"
I20260808 15:02:58.899024 10286 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 15:02:58.902163 10302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:58.903316 10286 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:02:58.903474 10286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "6d719451453945caae01fb4542a4fd16"
format_stamp: "Formatted at 2026-08-08 15:02:58 on dist-test-slave-jlzn"
server_key: "613f14f6e6f0c5d03238a2f48bb27696"
server_key_iv: "cd39e2ba642da6a3084c6b0d23930bc5"
server_key_version: "encryptionkey@0"
I20260808 15:02:58.903865 10286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:58.931352 10286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:58.932127 10286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:58.932403 10286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:58.949005 10286 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:40541
I20260808 15:02:58.949038 10376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:40541 every 8 connection(s)
I20260808 15:02:58.950187 10286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:58.953641 10378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:58.958583 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 10286
I20260808 15:02:58.958783 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:58.959381 2771 external_mini_cluster.cc:1428] Setting key 4b153edcccdaeffa181288dea1985cbc
I20260808 15:02:58.963752 10378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16: Bootstrap starting.
I20260808 15:02:58.966984 10378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:58.968237 10378 log.cc:824] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:58.971170 10378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16: No bootstrap required, opened a new log
I20260808 15:02:58.972621 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:58.961132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45556 (local address 127.2.180.254:40541)
0808 15:02:58.962080 (+ 948us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:58.962090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:58.962133 (+ 43us) server_negotiation.cc:407] Connection header received
0808 15:02:58.963730 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:58.963755 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:58.964216 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:58.964506 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:58.965161 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:58.966449 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:58.968327 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:58.969221 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:58.969409 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:58.970466 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:58.970473 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:58.970485 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:58.970997 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:58.971081 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:58.971102 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:58.971195 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:58.971431 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 15:02:58.971732 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":336,"threads_started":1}
I20260808 15:02:58.976151 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:02:58.979591 10378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d719451453945caae01fb4542a4fd16" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40541 } }
I20260808 15:02:58.979940 10378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:58.980033 10378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d719451453945caae01fb4542a4fd16, State: Initialized, Role: FOLLOWER
I20260808 15:02:58.980536 10378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d719451453945caae01fb4542a4fd16" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40541 } }
I20260808 15:02:58.980742 10378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:58.980870 10378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:58.981015 10378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:58.983045 10378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d719451453945caae01fb4542a4fd16" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40541 } }
I20260808 15:02:58.983500 10378 leader_election.cc:302] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d719451453945caae01fb4542a4fd16; no voters:
I20260808 15:02:58.984042 10378 leader_election.cc:288] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:58.984148 10386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:58.985198 10386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [term 1 LEADER]: Becoming Leader. State: Replica: 6d719451453945caae01fb4542a4fd16, State: Running, Role: LEADER
I20260808 15:02:58.985702 10386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d719451453945caae01fb4542a4fd16" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40541 } }
I20260808 15:02:58.985973 10378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:58.988094 10387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d719451453945caae01fb4542a4fd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d719451453945caae01fb4542a4fd16" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40541 } } }
I20260808 15:02:58.988448 10387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:58.988940 10389 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d719451453945caae01fb4542a4fd16. Latest consensus state: current_term: 1 leader_uuid: "6d719451453945caae01fb4542a4fd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d719451453945caae01fb4542a4fd16" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40541 } } }
I20260808 15:02:58.989233 10389 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16 [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:58.989904 10394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:58.994799 10394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:59.003039 10394 catalog_manager.cc:1347] Generated new cluster ID: 254e4daf533e479e85c37c4678f5012e
I20260808 15:02:59.003134 10394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:59.020809 10394 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:59.021929 10394 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:59.029819 10394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d719451453945caae01fb4542a4fd16: Generated new TSK 0
I20260808 15:02:59.030491 10394 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 15:02:59.074667 10421 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 15:02:59.074878 10421 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 15:02:59.075882 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:59.055744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45562 (local address 127.2.180.254:40541)
0808 15:02:59.055902 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:59.055908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:59.056418 (+ 510us) server_negotiation.cc:407] Connection header received
0808 15:02:59.056854 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:59.056862 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:59.056944 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:59.057084 (+ 140us) server_negotiation.cc:226] Negotiated authn=JWT
0808 15:02:59.057607 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.058307 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.059700 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.060204 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.060316 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:59.060796 (+ 480us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 15:02:59.075253 (+ 14457us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 15:02:59.075586 (+ 333us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45562: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 15:02:59.076004 10383 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45562: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 15:02:59.077235 10420 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:02:59.055678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40541 (local address 127.0.0.1:45562)
0808 15:02:59.056314 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:59.056333 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:59.056489 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:59.057152 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:59.057158 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:59.057170 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0808 15:02:59.057463 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:59.057472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.058441 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.058445 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:59.059558 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:59.059569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.060377 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.060385 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:59.060514 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:59.060567 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 15:02:59.075485 (+ 14918us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 15:02:59.075768 (+ 283us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.180.254:40541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":326,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.2.180.254:40541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46851/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 15:02:59.078477 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 10286
I20260808 15:02:59.079681 10371 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:59.171025 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 10286
2026-08-08T15:02:59Z chronyd exiting
I20260808 15:02:59.224198 2771 test_util.cc:175] -----------------------------------------------
I20260808 15:02:59.224339 2771 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786201338129629-2771-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (714 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 15:02:59.227982 2771 mini_oidc.cc:152] Starting JWKS server
I20260808 15:02:59.229735 2771 webserver.cc:460] Webserver started at https://127.0.0.1:32969/ using document root <none> and password file <none>
I20260808 15:02:59.229939 2771 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 15:02:59.230551 2771 webserver.cc:460] Webserver started at http://127.0.0.1:36327/ using document root <none> and password file <none>
2026-08-08T15:02:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:59Z Disabled control of system clock
I20260808 15:02:59.256227 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:33067
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:33067
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32969/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/testchainca.cert with env {}
W20260808 15:02:59.562101 10430 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:02:59.562389 10430 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:02:59.562517 10430 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:02:59.569429 10430 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:02:59.569535 10430 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:02:59.569595 10430 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:02:59.569672 10430 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:02:59.569736 10430 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 15:02:59.569818 10430 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 15:02:59.569869 10430 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32969/jwks.json
I20260808 15:02:59.579264 10430 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:33067
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:33067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32969/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:02:59.580430 10430 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:02:59.581622 10430 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:02:59.592665 10438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:59.592701 10441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:02:59.592615 10439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:02:59.593466 10430 server_base.cc:1034] running on GCE node
I20260808 15:02:59.594146 10430 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:02:59.595408 10430 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:02:59.596602 10430 hybrid_clock.cc:648] HybridClock initialized: now 1786201379596561 us; error 28 us; skew 500 ppm
I20260808 15:02:59.596905 10430 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:02:59.599004 10430 webserver.cc:460] Webserver started at http://127.2.180.254:41571/ using document root <none> and password file <none>
I20260808 15:02:59.599623 10430 fs_manager.cc:359] Metadata directory not provided
I20260808 15:02:59.599730 10430 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:02:59.600057 10430 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:02:59.602695 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "f7a54604a70f49ac86c25ce1b7b2308d"
format_stamp: "Formatted at 2026-08-08 15:02:59 on dist-test-slave-jlzn"
server_key: "9d0c1f2a30ad4977408b72f69b85e5d1"
server_key_iv: "33b72b0cdecb28d951ae609a0b0eb716"
server_key_version: "encryptionkey@0"
I20260808 15:02:59.603439 10430 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "f7a54604a70f49ac86c25ce1b7b2308d"
format_stamp: "Formatted at 2026-08-08 15:02:59 on dist-test-slave-jlzn"
server_key: "9d0c1f2a30ad4977408b72f69b85e5d1"
server_key_iv: "33b72b0cdecb28d951ae609a0b0eb716"
server_key_version: "encryptionkey@0"
I20260808 15:02:59.608661 10430 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 15:02:59.611783 10448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:02:59.612993 10430 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:02:59.613159 10430 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "f7a54604a70f49ac86c25ce1b7b2308d"
format_stamp: "Formatted at 2026-08-08 15:02:59 on dist-test-slave-jlzn"
server_key: "9d0c1f2a30ad4977408b72f69b85e5d1"
server_key_iv: "33b72b0cdecb28d951ae609a0b0eb716"
server_key_version: "encryptionkey@0"
I20260808 15:02:59.613349 10430 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:02:59.630208 10430 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:02:59.630930 10430 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:02:59.631202 10430 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:02:59.647744 10430 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:33067
I20260808 15:02:59.647775 10525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:33067 every 8 connection(s)
I20260808 15:02:59.649323 10430 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:02:59.650449 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 10430
I20260808 15:02:59.650651 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:02:59.651463 2771 external_mini_cluster.cc:1428] Setting key b72635001a87635d6aa158dcb1afcffb
I20260808 15:02:59.653790 10526 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:02:59.664521 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:59.653714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37084 (local address 127.2.180.254:33067)
0808 15:02:59.654660 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:59.654670 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:59.654710 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:02:59.656232 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:59.656253 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:59.656683 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:59.656998 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:02:59.657529 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.658817 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.660545 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.661147 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.661272 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:59.662409 (+ 1137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:02:59.662416 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:02:59.662427 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:02:59.662937 (+ 510us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:02:59.663007 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:02:59.663030 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:02:59.663123 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 15:02:59.663342 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 15:02:59.663622 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":355,"threads_started":1}
I20260808 15:02:59.666111 10526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d: Bootstrap starting.
I20260808 15:02:59.667981 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:02:59.669157 10526 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d: Neither blocks nor log segments found. Creating new log.
I20260808 15:02:59.670132 10526 log.cc:824] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d: Log is configured to *not* fsync() on all Append() calls
I20260808 15:02:59.672734 10526 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d: No bootstrap required, opened a new log
I20260808 15:02:59.679584 10526 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7a54604a70f49ac86c25ce1b7b2308d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33067 } }
I20260808 15:02:59.680022 10526 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:02:59.680126 10526 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7a54604a70f49ac86c25ce1b7b2308d, State: Initialized, Role: FOLLOWER
I20260808 15:02:59.680604 10526 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7a54604a70f49ac86c25ce1b7b2308d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33067 } }
I20260808 15:02:59.680816 10526 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:02:59.680936 10526 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:02:59.681097 10526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:02:59.683037 10526 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7a54604a70f49ac86c25ce1b7b2308d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33067 } }
I20260808 15:02:59.683495 10526 leader_election.cc:302] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7a54604a70f49ac86c25ce1b7b2308d; no voters:
I20260808 15:02:59.684026 10526 leader_election.cc:288] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:02:59.684126 10536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:02:59.684793 10536 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [term 1 LEADER]: Becoming Leader. State: Replica: f7a54604a70f49ac86c25ce1b7b2308d, State: Running, Role: LEADER
I20260808 15:02:59.685302 10536 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7a54604a70f49ac86c25ce1b7b2308d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33067 } }
I20260808 15:02:59.685832 10526 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:02:59.688025 10539 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7a54604a70f49ac86c25ce1b7b2308d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7a54604a70f49ac86c25ce1b7b2308d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33067 } } }
I20260808 15:02:59.688055 10541 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7a54604a70f49ac86c25ce1b7b2308d. Latest consensus state: current_term: 1 leader_uuid: "f7a54604a70f49ac86c25ce1b7b2308d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7a54604a70f49ac86c25ce1b7b2308d" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33067 } } }
I20260808 15:02:59.688565 10539 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:59.688637 10541 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d [sys.catalog]: This master's current role is: LEADER
I20260808 15:02:59.690263 10547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:02:59.694928 10547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:02:59.703060 10547 catalog_manager.cc:1347] Generated new cluster ID: 0f11969e5d7845349c8f2dd9b8f2e1fa
I20260808 15:02:59.703148 10547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:02:59.723348 10547 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:02:59.724534 10547 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:02:59.729597 10547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7a54604a70f49ac86c25ce1b7b2308d: Generated new TSK 0
I20260808 15:02:59.730247 10547 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 15:02:59.760435 10575 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 15:02:59.760536 10575 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260808 15:02:59.761724 10574 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:02:59.742363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:33067 (local address 127.0.0.1:37098)
0808 15:02:59.742892 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 15:02:59.742907 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:02:59.743018 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:02:59.743403 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:02:59.743406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:02:59.743414 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 15:02:59.743655 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:59.743663 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.745034 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.745038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:59.746213 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:02:59.746224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.747254 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.747259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:02:59.747387 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:59.747417 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 15:02:59.761261 (+ 13844us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 15:02:59.761430 (+ 169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.180.254:33067: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":288,"threads_started":1}
I20260808 15:02:59.761721 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:02:59.742502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.2.180.254:33067)
0808 15:02:59.742683 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 15:02:59.742690 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:02:59.742982 (+ 292us) server_negotiation.cc:407] Connection header received
0808 15:02:59.743174 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:02:59.743179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:02:59.743257 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:02:59.743406 (+ 149us) server_negotiation.cc:226] Negotiated authn=JWT
0808 15:02:59.743822 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.744915 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.746349 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:02:59.747085 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:02:59.747220 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:02:59.747645 (+ 425us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 15:02:59.761068 (+ 13423us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 15:02:59.761400 (+ 332us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37098: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 15:02:59.761829 10531 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37098: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32969/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260808 15:02:59.762653 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 10430
I20260808 15:02:59.763734 10519 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:02:59.853025 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 10430
2026-08-08T15:02:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (682 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 15:02:59.910585 2771 mini_oidc.cc:152] Starting JWKS server
I20260808 15:02:59.912402 2771 webserver.cc:460] Webserver started at https://127.0.0.1:38109/ using document root <none> and password file <none>
I20260808 15:02:59.912650 2771 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 15:02:59.913225 2771 webserver.cc:460] Webserver started at http://127.0.0.1:36421/ using document root <none> and password file <none>
2026-08-08T15:02:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:02:59Z Disabled control of system clock
I20260808 15:02:59.940786 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:42355
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:34699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:42355
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38109/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 15:03:00.245697 10587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:00.245965 10587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:00.246091 10587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:00.253062 10587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:00.253167 10587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:00.253232 10587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:00.253306 10587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:00.253367 10587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 15:03:00.253449 10587 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 15:03:00.253502 10587 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38109/jwks.json
I20260808 15:03:00.262082 10587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:34699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:42355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:42355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38109/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:00.263212 10587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:00.264916 10587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:00.275717 10592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:00.275672 10594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:00.275839 10598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:00.276710 10587 server_base.cc:1034] running on GCE node
I20260808 15:03:00.277383 10587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:00.278594 10587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:00.279817 10587 hybrid_clock.cc:648] HybridClock initialized: now 1786201380279754 us; error 34 us; skew 500 ppm
I20260808 15:03:00.280130 10587 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:00.282178 10587 webserver.cc:460] Webserver started at http://127.2.180.254:34465/ using document root <none> and password file <none>
I20260808 15:03:00.282786 10587 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:00.282882 10587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:00.283182 10587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:00.285898 10587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "f3122df2fd994eeab54134a6988fe389"
format_stamp: "Formatted at 2026-08-08 15:03:00 on dist-test-slave-jlzn"
server_key: "c44275eec3bc1964be205516dd547fc3"
server_key_iv: "10d790d8a72f1880edaa4a380a0bff1f"
server_key_version: "encryptionkey@0"
I20260808 15:03:00.286628 10587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "f3122df2fd994eeab54134a6988fe389"
format_stamp: "Formatted at 2026-08-08 15:03:00 on dist-test-slave-jlzn"
server_key: "c44275eec3bc1964be205516dd547fc3"
server_key_iv: "10d790d8a72f1880edaa4a380a0bff1f"
server_key_version: "encryptionkey@0"
I20260808 15:03:00.291256 10587 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:03:00.294368 10606 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:00.295534 10587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:03:00.295689 10587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "f3122df2fd994eeab54134a6988fe389"
format_stamp: "Formatted at 2026-08-08 15:03:00 on dist-test-slave-jlzn"
server_key: "c44275eec3bc1964be205516dd547fc3"
server_key_iv: "10d790d8a72f1880edaa4a380a0bff1f"
server_key_version: "encryptionkey@0"
I20260808 15:03:00.295907 10587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:00.326951 10587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:00.327689 10587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:00.327996 10587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:00.344481 10587 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:42355
I20260808 15:03:00.344498 10680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:42355 every 8 connection(s)
I20260808 15:03:00.345707 10587 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:00.349256 10681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:00.354959 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 10587
I20260808 15:03:00.355157 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:00.355743 2771 external_mini_cluster.cc:1428] Setting key ee685fc4e996334e940a7f3cf77e55e9
I20260808 15:03:00.359231 10681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389: Bootstrap starting.
I20260808 15:03:00.362324 10681 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:00.363502 10681 log.cc:824] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:00.366165 10681 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389: No bootstrap required, opened a new log
I20260808 15:03:00.368431 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:00.357513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.2.180.254:42355)
0808 15:03:00.358407 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:00.358418 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:00.358457 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:03:00.360071 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:00.360092 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:00.360494 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:00.360792 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:00.361340 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:00.362586 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:00.364318 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:00.364973 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:00.365162 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:00.366280 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:00.366289 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:00.366302 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:00.366802 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:00.366876 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:00.366896 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:00.367001 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:00.367220 (+ 219us) server_negotiation.cc:299] Negotiation successful
0808 15:03:00.367487 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":510,"thread_start_us":307,"threads_started":1}
I20260808 15:03:00.372215 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:03:00.373944 10681 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3122df2fd994eeab54134a6988fe389" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42355 } }
I20260808 15:03:00.374284 10681 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:00.374401 10681 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3122df2fd994eeab54134a6988fe389, State: Initialized, Role: FOLLOWER
I20260808 15:03:00.374918 10681 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3122df2fd994eeab54134a6988fe389" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42355 } }
I20260808 15:03:00.375138 10681 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:00.375299 10681 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:00.375435 10681 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:00.377395 10681 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3122df2fd994eeab54134a6988fe389" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42355 } }
I20260808 15:03:00.377837 10681 leader_election.cc:302] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3122df2fd994eeab54134a6988fe389; no voters:
I20260808 15:03:00.378348 10681 leader_election.cc:288] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:00.378451 10687 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:00.379115 10687 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [term 1 LEADER]: Becoming Leader. State: Replica: f3122df2fd994eeab54134a6988fe389, State: Running, Role: LEADER
I20260808 15:03:00.379619 10687 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3122df2fd994eeab54134a6988fe389" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42355 } }
I20260808 15:03:00.380115 10681 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:00.382015 10688 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3122df2fd994eeab54134a6988fe389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3122df2fd994eeab54134a6988fe389" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42355 } } }
I20260808 15:03:00.382308 10688 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:00.382807 10689 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3122df2fd994eeab54134a6988fe389. Latest consensus state: current_term: 1 leader_uuid: "f3122df2fd994eeab54134a6988fe389" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3122df2fd994eeab54134a6988fe389" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42355 } } }
I20260808 15:03:00.383114 10689 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:00.383132 10696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:00.388723 10696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:00.396981 10696 catalog_manager.cc:1347] Generated new cluster ID: 50aa74be457c41669d7bdde546a68ca0
I20260808 15:03:00.397074 10696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:00.415822 10696 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:00.417006 10696 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:00.425717 10696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3122df2fd994eeab54134a6988fe389: Generated new TSK 0
I20260808 15:03:00.426403 10696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:03:00.446319 10685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:00.438477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.2.180.254:42355)
0808 15:03:00.438661 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:00.438668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:00.438984 (+ 316us) server_negotiation.cc:407] Connection header received
0808 15:03:00.439181 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:00.439187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:00.439272 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:00.439415 (+ 143us) server_negotiation.cc:226] Negotiated authn=JWT
0808 15:03:00.439841 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:00.440962 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:00.442292 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:00.442987 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:00.443110 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:00.443507 (+ 397us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 15:03:00.445699 (+ 2192us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 15:03:00.446010 (+ 311us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40194: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":64}
W20260808 15:03:00.446434 10685 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40194: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 15:03:00.448170 10730 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:00.438361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:42355 (local address 127.0.0.1:40194)
0808 15:03:00.438891 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:00.438907 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:00.439016 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:00.439413 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:00.439416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:00.439424 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 15:03:00.439673 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:00.439680 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:00.441096 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:00.441099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:00.442160 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:00.442171 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:00.443142 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:00.443146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:00.443265 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:00.443294 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 15:03:00.445902 (+ 2608us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 15:03:00.447753 (+ 1851us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.180.254:42355: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38109/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":1566,"thread_start_us":290,"threads_started":1}
I20260808 15:03:00.449499 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 10587
I20260808 15:03:00.450594 10672 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 15:03:00.458376 10733 webserver.cc:533] Webserver: sslize failed:
E20260808 15:03:00.458455 10733 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 15:03:00.538982 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 10587
2026-08-08T15:03:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (688 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T15:03:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:00Z Disabled control of system clock
I20260808 15:03:00.626084 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:34151
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:34635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:34151
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/testchainca.cert with env {}
W20260808 15:03:00.937697 10742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:00.937945 10742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:00.938068 10742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:00.944928 10742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:00.945019 10742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:00.945081 10742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:00.945158 10742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:00.945218 10742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 15:03:00.945302 10742 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 15:03:00.945353 10742 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 15:03:00.954713 10742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:34635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:34151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:34151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:00.955834 10742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:00.957038 10742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:00.968052 10748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:00.968101 10750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:00.968134 10754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:00.968859 10742 server_base.cc:1034] running on GCE node
I20260808 15:03:00.969583 10742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:00.970847 10742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:00.972043 10742 hybrid_clock.cc:648] HybridClock initialized: now 1786201380971995 us; error 39 us; skew 500 ppm
I20260808 15:03:00.972359 10742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:00.974395 10742 webserver.cc:460] Webserver started at http://127.2.180.254:33455/ using document root <none> and password file <none>
I20260808 15:03:00.975039 10742 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:00.975148 10742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:00.975450 10742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:00.978139 10742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "6c9e51d122c6499390df061ce5c24beb"
format_stamp: "Formatted at 2026-08-08 15:03:00 on dist-test-slave-jlzn"
server_key: "7ed3598ca4ecefa871fe2dbecd03cd41"
server_key_iv: "4c479c0677bf351c32683ee133971d16"
server_key_version: "encryptionkey@0"
I20260808 15:03:00.978893 10742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "6c9e51d122c6499390df061ce5c24beb"
format_stamp: "Formatted at 2026-08-08 15:03:00 on dist-test-slave-jlzn"
server_key: "7ed3598ca4ecefa871fe2dbecd03cd41"
server_key_iv: "4c479c0677bf351c32683ee133971d16"
server_key_version: "encryptionkey@0"
I20260808 15:03:00.983501 10742 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 15:03:00.986655 10764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:00.987779 10742 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:03:00.987993 10742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "6c9e51d122c6499390df061ce5c24beb"
format_stamp: "Formatted at 2026-08-08 15:03:00 on dist-test-slave-jlzn"
server_key: "7ed3598ca4ecefa871fe2dbecd03cd41"
server_key_iv: "4c479c0677bf351c32683ee133971d16"
server_key_version: "encryptionkey@0"
I20260808 15:03:00.988185 10742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:01.005059 10742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:01.005745 10742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:01.005980 10742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:01.021551 10742 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:34151
I20260808 15:03:01.021579 10838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:34151 every 8 connection(s)
I20260808 15:03:01.022722 10742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:01.026101 10839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:01.032081 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 10742
I20260808 15:03:01.032243 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:01.032850 2771 external_mini_cluster.cc:1428] Setting key 54f973a68ec6c5825bd40794e729e76b
I20260808 15:03:01.036135 10839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb: Bootstrap starting.
I20260808 15:03:01.039951 10839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:01.041157 10839 log.cc:824] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:01.044119 10839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb: No bootstrap required, opened a new log
I20260808 15:03:01.047616 10844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:01.034524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.2.180.254:34151)
0808 15:03:01.035540 (+ 1016us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:01.035553 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:01.035602 (+ 49us) server_negotiation.cc:407] Connection header received
0808 15:03:01.037899 (+ 2297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:01.037926 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:01.038467 (+ 541us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:01.038847 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:01.039351 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:01.041294 (+ 1943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:01.042870 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:01.043867 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:01.044041 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:01.044995 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:01.045006 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:01.045022 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:01.045681 (+ 659us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:01.045774 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:01.045809 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:01.045912 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:01.046106 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 15:03:01.046393 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":317,"threads_started":1}
I20260808 15:03:01.051450 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:03:01.052923 10839 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9e51d122c6499390df061ce5c24beb" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34151 } }
I20260808 15:03:01.053215 10839 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:01.053303 10839 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c9e51d122c6499390df061ce5c24beb, State: Initialized, Role: FOLLOWER
I20260808 15:03:01.053917 10839 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9e51d122c6499390df061ce5c24beb" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34151 } }
I20260808 15:03:01.054181 10839 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:01.054327 10839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:01.054513 10839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:01.056802 10839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9e51d122c6499390df061ce5c24beb" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34151 } }
I20260808 15:03:01.057210 10839 leader_election.cc:302] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c9e51d122c6499390df061ce5c24beb; no voters:
I20260808 15:03:01.057709 10839 leader_election.cc:288] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:01.057822 10846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:01.058473 10846 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [term 1 LEADER]: Becoming Leader. State: Replica: 6c9e51d122c6499390df061ce5c24beb, State: Running, Role: LEADER
I20260808 15:03:01.058965 10846 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9e51d122c6499390df061ce5c24beb" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34151 } }
I20260808 15:03:01.059439 10839 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:01.061673 10849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c9e51d122c6499390df061ce5c24beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9e51d122c6499390df061ce5c24beb" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34151 } } }
I20260808 15:03:01.061705 10852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c9e51d122c6499390df061ce5c24beb. Latest consensus state: current_term: 1 leader_uuid: "6c9e51d122c6499390df061ce5c24beb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c9e51d122c6499390df061ce5c24beb" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 34151 } } }
I20260808 15:03:01.062119 10849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:01.062116 10852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:01.063709 10861 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:01.068965 10861 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:01.077543 10861 catalog_manager.cc:1347] Generated new cluster ID: 1012951a720d4704b75048fba70d6bcd
I20260808 15:03:01.077677 10861 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:01.091638 10861 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:01.092887 10861 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:01.100003 10861 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c9e51d122c6499390df061ce5c24beb: Generated new TSK 0
I20260808 15:03:01.100625 10861 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:03:01.202205 10844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:01.115344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36224 (local address 127.2.180.254:34151)
0808 15:03:01.115528 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:01.115533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:01.115868 (+ 335us) server_negotiation.cc:407] Connection header received
0808 15:03:01.116077 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:01.116081 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:01.116151 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:01.116269 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0808 15:03:01.116783 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:01.117490 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:01.119456 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:01.119932 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:01.120020 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:01.120544 (+ 524us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 15:03:01.201619 (+ 81075us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 15:03:01.201943 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36224: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 15:03:01.202340 10844 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36224: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 15:03:01.202659 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:01.115159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:34151 (local address 127.0.0.1:36224)
0808 15:03:01.115748 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:01.115768 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:01.115936 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:01.116296 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:01.116301 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:01.116314 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0808 15:03:01.116640 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:01.116654 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:01.117631 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:01.117637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:01.119324 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:01.119339 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:01.120098 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:01.120105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:01.120297 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:01.120342 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 15:03:01.201918 (+ 81576us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 15:03:01.202256 (+ 338us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.2.180.254:34151: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":312,"threads_started":1}
I20260808 15:03:01.204349 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 10742
I20260808 15:03:01.205449 10833 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:01.302359 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 10742
2026-08-08T15:03:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (743 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (394 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (397 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:03:02 dist-test-slave-jlzn krb5kdc[10899](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:03:02 dist-test-slave-jlzn krb5kdc[10899](info): set up 2 sockets
Aug 08 15:03:02 dist-test-slave-jlzn krb5kdc[10899](info): commencing operation
krb5kdc: starting...
W20260808 15:03:03.272959 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:03:03 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:03:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:03:03.457440 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:42089
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:42089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:03.766858 10917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:03.767120 10917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:03.767233 10917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:03.774194 10917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:03.774297 10917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:03.774369 10917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:03.774439 10917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:03.774498 10917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:03.783905 10917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:42089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:42089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:03.785104 10917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:03.786357 10917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:03.797636 10922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:03.797660 10923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:03.797870 10928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:03.798326 10917 server_base.cc:1034] running on GCE node
I20260808 15:03:03.799059 10917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:03.800451 10917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:03.801651 10917 hybrid_clock.cc:648] HybridClock initialized: now 1786201383801603 us; error 35 us; skew 500 ppm
I20260808 15:03:03.801992 10917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:03 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201383, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:03.804975 10917 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:03:03.806318 10917 webserver.cc:460] Webserver started at http://127.2.180.254:36895/ using document root <none> and password file <none>
I20260808 15:03:03.806959 10917 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:03.807071 10917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:03.807384 10917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:03.810314 10917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd"
format_stamp: "Formatted at 2026-08-08 15:03:03 on dist-test-slave-jlzn"
server_key: "9b53d2331f77d480a892d88391bad954"
server_key_iv: "c5af5ec3edc5f43d7afee2a9b8993ef5"
server_key_version: "encryptionkey@0"
I20260808 15:03:03.811112 10917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd"
format_stamp: "Formatted at 2026-08-08 15:03:03 on dist-test-slave-jlzn"
server_key: "9b53d2331f77d480a892d88391bad954"
server_key_iv: "c5af5ec3edc5f43d7afee2a9b8993ef5"
server_key_version: "encryptionkey@0"
I20260808 15:03:03.816144 10917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 15:03:03.819415 10941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:03.820663 10917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:03.820820 10917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd"
format_stamp: "Formatted at 2026-08-08 15:03:03 on dist-test-slave-jlzn"
server_key: "9b53d2331f77d480a892d88391bad954"
server_key_iv: "c5af5ec3edc5f43d7afee2a9b8993ef5"
server_key_version: "encryptionkey@0"
I20260808 15:03:03.820990 10917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:03.850905 10917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:03.854738 10917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:03.854997 10917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:03.871887 10917 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:42089
I20260808 15:03:03.871959 11020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:42089 every 8 connection(s)
I20260808 15:03:03.873159 10917 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:03.876720 11021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:03.883553 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 10917
I20260808 15:03:03.883776 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:03.884572 2771 external_mini_cluster.cc:1428] Setting key b179f819355dfeaa82b8f2a9bb90f37e
I20260808 15:03:03.887427 11021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd: Bootstrap starting.
I20260808 15:03:03.890709 11021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:03.891924 11021 log.cc:824] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:03.894999 11021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd: No bootstrap required, opened a new log
Aug 08 15:03:03 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:03.904701 11021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42089 } }
I20260808 15:03:03.905119 11021 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:03.905236 11021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d32cc369eae4d5c8731f6ca3c61d3cd, State: Initialized, Role: FOLLOWER
I20260808 15:03:03.905889 11021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42089 } }
I20260808 15:03:03.906194 11021 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:03.906354 11021 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:03.906565 11021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:03.908048 11026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:03.886467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.2.180.254:42089)
0808 15:03:03.887206 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:03.887217 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:03.887258 (+ 41us) server_negotiation.cc:407] Connection header received
0808 15:03:03.889033 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:03.889054 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:03.889409 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:03.889696 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:03.891388 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:03.893036 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:03.894745 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:03.895433 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:03.895570 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:03.899905 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:03.899915 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:03.899932 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:03.899985 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:03.903770 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:03.904401 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:03.904409 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:03.904416 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:03.904628 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:03.904956 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:03.904961 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:03.904964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:03.905666 (+ 702us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:03.905963 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:03.906289 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 15:03:03.906729 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":342,"threads_started":1}
I20260808 15:03:03.909289 11021 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42089 } }
I20260808 15:03:03.909739 11021 leader_election.cc:302] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d32cc369eae4d5c8731f6ca3c61d3cd; no voters:
I20260808 15:03:03.910313 11021 leader_election.cc:288] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:03.910409 11028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:03.911355 11028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [term 1 LEADER]: Becoming Leader. State: Replica: 6d32cc369eae4d5c8731f6ca3c61d3cd, State: Running, Role: LEADER
I20260808 15:03:03.911834 11028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42089 } }
I20260808 15:03:03.912297 11021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:03.914295 11030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42089 } } }
I20260808 15:03:03.914646 11030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:03.916046 11031 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d32cc369eae4d5c8731f6ca3c61d3cd. Latest consensus state: current_term: 1 leader_uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d32cc369eae4d5c8731f6ca3c61d3cd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 42089 } } }
I20260808 15:03:03.916343 11031 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:03.917255 11035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:03.921959 11035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:03.931031 11035 catalog_manager.cc:1347] Generated new cluster ID: 1a1ebeb5739e44aaa5e08f6e0d3ae3bd
I20260808 15:03:03.931180 11035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:03.986910 11035 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:03.988212 11035 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:03.995718 11035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d32cc369eae4d5c8731f6ca3c61d3cd: Generated new TSK 0
I20260808 15:03:03.996443 11035 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:03:04.054980 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42089
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:04.433025 11055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:04.433311 11055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:04.433445 11055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:04.440397 11055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:04.440502 11055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:04.440650 11055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:04.449793 11055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:04.450968 11055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:04.452231 11055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:04.464389 11060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:04.464434 11061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:04.464363 11063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:04.465173 11055 server_base.cc:1034] running on GCE node
I20260808 15:03:04.465700 11055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:04.466493 11055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:04.467684 11055 hybrid_clock.cc:648] HybridClock initialized: now 1786201384467635 us; error 38 us; skew 500 ppm
I20260808 15:03:04.467979 11055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:04 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201384, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:04.470887 11055 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:03:04.472190 11055 webserver.cc:460] Webserver started at http://127.2.180.193:37637/ using document root <none> and password file <none>
I20260808 15:03:04.472766 11055 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:04.472865 11055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:04.473101 11055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:04.475821 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "35db02904a3f48f3a20ec189e03b0f1b"
format_stamp: "Formatted at 2026-08-08 15:03:04 on dist-test-slave-jlzn"
server_key: "8286db6ba86a8c4436b23b83243b60da"
server_key_iv: "a493635bf9a98d900da5481403de7b96"
server_key_version: "encryptionkey@0"
I20260808 15:03:04.476557 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "35db02904a3f48f3a20ec189e03b0f1b"
format_stamp: "Formatted at 2026-08-08 15:03:04 on dist-test-slave-jlzn"
server_key: "8286db6ba86a8c4436b23b83243b60da"
server_key_iv: "a493635bf9a98d900da5481403de7b96"
server_key_version: "encryptionkey@0"
I20260808 15:03:04.481205 11055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 15:03:04.484277 11072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:04.485484 11055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260808 15:03:04.485642 11055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "35db02904a3f48f3a20ec189e03b0f1b"
format_stamp: "Formatted at 2026-08-08 15:03:04 on dist-test-slave-jlzn"
server_key: "8286db6ba86a8c4436b23b83243b60da"
server_key_iv: "a493635bf9a98d900da5481403de7b96"
server_key_version: "encryptionkey@0"
I20260808 15:03:04.485824 11055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:04.519374 11055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:04.523361 11055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:04.523617 11055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:04.524600 11055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:04.525918 11055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:04.526010 11055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:04.526121 11055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:04.526172 11055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:04.554955 11055 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:39721
I20260808 15:03:04.555024 11236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:39721 every 8 connection(s)
I20260808 15:03:04.556185 11055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:04.556835 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 11055
I20260808 15:03:04.557065 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:03:04.557724 2771 external_mini_cluster.cc:1428] Setting key a8acf1418240a66e1c9811a90e114af0
Aug 08 15:03:04 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201384, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:04.578327 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:04.558881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:59725 (local address 127.2.180.254:42089)
0808 15:03:04.559324 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:04.559328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:04.562907 (+ 3579us) server_negotiation.cc:407] Connection header received
0808 15:03:04.562972 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:04.562977 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:04.563050 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:04.563143 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:04.565128 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:04.566125 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:04.568199 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:04.568743 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:04.568840 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:04.572447 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:04.572453 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:04.572458 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:04.572507 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:04.575283 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:04.576159 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:04.576162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:04.576165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:04.576284 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:04.576736 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:04.576740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:04.576742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:04.576999 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:04.577131 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:04.577911 (+ 780us) server_negotiation.cc:299] Negotiation successful
0808 15:03:04.578081 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":242,"threads_started":1}
I20260808 15:03:04.579521 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:04.559515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:42089 (local address 127.2.180.193:59725)
0808 15:03:04.560383 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:04.560431 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:04.562583 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:04.563540 (+ 957us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:04.563550 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:04.564230 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:04.564940 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:04.564963 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:04.566334 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:04.566338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:04.568035 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:04.568049 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:04.569133 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:04.569138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:04.569329 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:04.569923 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:04.569950 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:04.572140 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:04.575419 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:04.575427 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:04.575441 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:04.576049 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:04.576381 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:04.576386 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:04.576389 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:04.576611 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:04.577143 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:04.577151 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:04.577526 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 15:03:04.577801 (+ 275us) client_negotiation.cc:240] Negotiation successful
0808 15:03:04.578254 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":368,"threads_started":1}
I20260808 15:03:04.581074 11239 heartbeater.cc:347] Connected to a master server at 127.2.180.254:42089
I20260808 15:03:04.581354 11239 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:04.582049 11239 heartbeater.cc:511] Master 127.2.180.254:42089 requested a full tablet report, sending...
I20260808 15:03:04.584069 10963 ts_manager.cc:194] Registered new tserver with Master: 35db02904a3f48f3a20ec189e03b0f1b (127.2.180.193:39721)
I20260808 15:03:04.585443 10963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:59725
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:03:04.610848 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42089
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:04.919344 11248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:04.919605 11248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:04.919742 11248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:04.926744 11248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:04.926848 11248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:04.926991 11248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:04.937115 11248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:04.938223 11248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:04.939548 11248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:04.951611 11254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:04.951542 11255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:04.951576 11258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:04.952566 11248 server_base.cc:1034] running on GCE node
I20260808 15:03:04.953143 11248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:04.954000 11248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:04.955195 11248 hybrid_clock.cc:648] HybridClock initialized: now 1786201384955151 us; error 31 us; skew 500 ppm
I20260808 15:03:04.955476 11248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:04 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201384, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:04.958356 11248 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:03:04.959651 11248 webserver.cc:460] Webserver started at http://127.2.180.194:37405/ using document root <none> and password file <none>
I20260808 15:03:04.960254 11248 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:04.960356 11248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:04.960608 11248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:04.963220 11248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "129ffd2ad22e479aacaa599cac9bdcd5"
format_stamp: "Formatted at 2026-08-08 15:03:04 on dist-test-slave-jlzn"
server_key: "e4928c586e7df1555795679fd6a1c002"
server_key_iv: "4ed9a99b2aacb14c7490c96bfd584571"
server_key_version: "encryptionkey@0"
I20260808 15:03:04.963985 11248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "129ffd2ad22e479aacaa599cac9bdcd5"
format_stamp: "Formatted at 2026-08-08 15:03:04 on dist-test-slave-jlzn"
server_key: "e4928c586e7df1555795679fd6a1c002"
server_key_iv: "4ed9a99b2aacb14c7490c96bfd584571"
server_key_version: "encryptionkey@0"
I20260808 15:03:04.968576 11248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:03:04.971604 11270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:04.972795 11248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:03:04.972915 11248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "129ffd2ad22e479aacaa599cac9bdcd5"
format_stamp: "Formatted at 2026-08-08 15:03:04 on dist-test-slave-jlzn"
server_key: "e4928c586e7df1555795679fd6a1c002"
server_key_iv: "4ed9a99b2aacb14c7490c96bfd584571"
server_key_version: "encryptionkey@0"
I20260808 15:03:04.973068 11248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:04.994289 11248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:04.997860 11248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:04.998073 11248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:04.998864 11248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:05.000039 11248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:05.000115 11248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:05.000200 11248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:05.000241 11248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:05.029491 11248 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:33643
I20260808 15:03:05.029552 11448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:33643 every 8 connection(s)
I20260808 15:03:05.030611 11248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:05.037390 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 11248
I20260808 15:03:05.037542 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:03:05.037940 2771 external_mini_cluster.cc:1428] Setting key ceb8a6724457db7f7dbf4db5fc8bea28
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201384, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:05.049438 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.032867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:34751 (local address 127.2.180.254:42089)
0808 15:03:05.033043 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.033047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.034117 (+ 1070us) server_negotiation.cc:407] Connection header received
0808 15:03:05.035860 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.035866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.035946 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.036036 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.037455 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.038234 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.039915 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.040354 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.040463 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.044000 (+ 3537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:05.044005 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:05.044008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:05.044041 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:05.046458 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.047250 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.047254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.047256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.047382 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.047837 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.047842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.047845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.048179 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:05.048321 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:05.048909 (+ 588us) server_negotiation.cc:299] Negotiation successful
0808 15:03:05.049141 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:03:05.050105 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.033297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:42089 (local address 127.2.180.194:34751)
0808 15:03:05.033975 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:05.034010 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:05.035583 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:05.036261 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:05.036269 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:05.036740 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:05.037291 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:05.037309 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.038380 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.038384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:05.039764 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:05.039777 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.040572 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.040581 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:05.040795 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.041401 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:05.041423 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:05.043751 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:05.046599 (+ 2848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:05.046607 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:05.046620 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:05.047144 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:05.047475 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:05.047479 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:05.047482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:05.047722 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:05.048290 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:05.048295 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:05.048570 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 15:03:05.048788 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 15:03:05.049127 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":279,"threads_started":1}
I20260808 15:03:05.051353 11451 heartbeater.cc:347] Connected to a master server at 127.2.180.254:42089
I20260808 15:03:05.051671 11451 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:05.052541 11451 heartbeater.cc:511] Master 127.2.180.254:42089 requested a full tablet report, sending...
I20260808 15:03:05.054056 10963 ts_manager.cc:194] Registered new tserver with Master: 129ffd2ad22e479aacaa599cac9bdcd5 (127.2.180.194:33643)
I20260808 15:03:05.054893 10963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:34751
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:03:05.084486 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42089
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:05.393618 11456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:05.393851 11456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:05.393972 11456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:05.400812 11456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:05.400908 11456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:05.401059 11456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:05.410127 11456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:42089
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:05.411244 11456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:05.412508 11456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:05.424543 11469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:05.424573 11467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:05.424674 11466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:05.425325 11456 server_base.cc:1034] running on GCE node
I20260808 15:03:05.425863 11456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:05.426712 11456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:05.427889 11456 hybrid_clock.cc:648] HybridClock initialized: now 1786201385427852 us; error 29 us; skew 500 ppm
I20260808 15:03:05.428153 11456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:05.430969 11456 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:03:05.432397 11456 webserver.cc:460] Webserver started at http://127.2.180.195:42887/ using document root <none> and password file <none>
I20260808 15:03:05.433000 11456 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:05.433109 11456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:05.433364 11456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:05.436168 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "a48788b4d89949df83895f35bd5e44ae"
format_stamp: "Formatted at 2026-08-08 15:03:05 on dist-test-slave-jlzn"
server_key: "ce0a208146e961ad74936adc78e5af31"
server_key_iv: "43009741feb070d7a762d190fc1ee282"
server_key_version: "encryptionkey@0"
I20260808 15:03:05.436882 11456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "a48788b4d89949df83895f35bd5e44ae"
format_stamp: "Formatted at 2026-08-08 15:03:05 on dist-test-slave-jlzn"
server_key: "ce0a208146e961ad74936adc78e5af31"
server_key_iv: "43009741feb070d7a762d190fc1ee282"
server_key_version: "encryptionkey@0"
I20260808 15:03:05.441577 11456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:03:05.444756 11477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:05.445942 11456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:05.446075 11456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "a48788b4d89949df83895f35bd5e44ae"
format_stamp: "Formatted at 2026-08-08 15:03:05 on dist-test-slave-jlzn"
server_key: "ce0a208146e961ad74936adc78e5af31"
server_key_iv: "43009741feb070d7a762d190fc1ee282"
server_key_version: "encryptionkey@0"
I20260808 15:03:05.446240 11456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:05.465540 11456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:05.469012 11456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:05.469233 11456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:05.470055 11456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:05.471323 11456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:05.471395 11456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:05.471484 11456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:05.471547 11456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:05.501075 11456 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:46407
I20260808 15:03:05.501127 11638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:46407 every 8 connection(s)
I20260808 15:03:05.502310 11456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:05.509583 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 11456
I20260808 15:03:05.509745 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:03:05.510162 2771 external_mini_cluster.cc:1428] Setting key e4200aab6cc34b875eb940f652cf851b
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:05.520767 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.504798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:33667 (local address 127.2.180.254:42089)
0808 15:03:05.504982 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.504986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.505980 (+ 994us) server_negotiation.cc:407] Connection header received
0808 15:03:05.507711 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.507717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.507862 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.507948 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.509397 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.510206 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.512032 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.512491 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.512606 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.515748 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:05.515753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:05.515756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:05.515788 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:05.518249 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.518907 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.518910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.518912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.519033 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.519401 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.519404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.519406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.519648 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:05.519752 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:05.520366 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 15:03:05.520521 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 15:03:05.521631 11646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.505131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:42089 (local address 127.2.180.195:33667)
0808 15:03:05.505829 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:05.505876 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:05.507463 (+ 1587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:05.508194 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:05.508201 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:05.508679 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:05.509220 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:05.509246 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.510356 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.510360 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:05.511895 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:05.511908 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.512720 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.512727 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:05.512948 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.513514 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:05.513540 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:05.515558 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:05.518357 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:05.518364 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:05.518374 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:05.518823 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:05.519116 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:05.519120 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:05.519123 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:05.519324 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:05.519736 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:05.519741 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:05.520054 (+ 313us) client_negotiation.cc:769] Sending connection context
0808 15:03:05.520280 (+ 226us) client_negotiation.cc:240] Negotiation successful
0808 15:03:05.520625 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":280,"threads_started":1}
I20260808 15:03:05.522842 11639 heartbeater.cc:347] Connected to a master server at 127.2.180.254:42089
I20260808 15:03:05.523100 11639 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:05.523857 11639 heartbeater.cc:511] Master 127.2.180.254:42089 requested a full tablet report, sending...
I20260808 15:03:05.525297 10963 ts_manager.cc:194] Registered new tserver with Master: a48788b4d89949df83895f35bd5e44ae (127.2.180.195:46407)
I20260808 15:03:05.526149 10963 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:33667
I20260808 15:03:05.536705 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/test-admin.keytab.
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:05.564728 2771 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 15:03:05.569389 11656 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:05.567091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57360 (local address 127.2.180.193:39721)
0808 15:03:05.567682 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.567688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.567711 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:03:05.567787 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.567792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.567997 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.568145 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.568982 (+ 837us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57360: BlockingRecv error: recv got EOF from 127.0.0.1:57360 (error 108)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":313,"threads_started":1}
W20260808 15:03:05.569734 2771 init.cc:244] Error while opening credential cache '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5cc)
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:05.571079 2771 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:05.572309 2771 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:03:05.585896 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.573679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57364 (local address 127.2.180.193:39721)
0808 15:03:05.573866 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.573872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.574070 (+ 198us) server_negotiation.cc:407] Connection header received
0808 15:03:05.574187 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.574190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.574244 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.574319 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.575217 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.576187 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.577419 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.577924 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.578013 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.581166 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:05.581177 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:05.581190 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:05.581227 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:05.583655 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.584257 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.584264 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.584266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.584396 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.584672 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.584675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.584677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.584992 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:05.585200 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:05.585410 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 15:03:05.585638 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:03:05.588006 11239 heartbeater.cc:503] Master 127.2.180.254:42089 was elected leader, sending a full tablet report...
W20260808 15:03:05.591120 11656 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:05.589621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57378 (local address 127.2.180.193:39721)
0808 15:03:05.589813 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.589818 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.590018 (+ 200us) server_negotiation.cc:407] Connection header received
0808 15:03:05.590146 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.590158 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.590212 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.590281 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.590929 (+ 648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57378: BlockingRecv error: recv got EOF from 127.0.0.1:57378 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
W20260808 15:03:05.591289 2771 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5cc' of type FILE
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:05.592675 2771 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:05.593847 2771 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:03:05.606114 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.594882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.2.180.193:39721)
0808 15:03:05.595007 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.595010 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.595315 (+ 305us) server_negotiation.cc:407] Connection header received
0808 15:03:05.595431 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.595436 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.595492 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.595564 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.596426 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.597070 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.598136 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.598536 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.598628 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.601654 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:05.601659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:05.601662 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:05.601700 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:05.603951 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.604507 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.604512 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.604515 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.604628 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.604963 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.604968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.604971 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.605224 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:05.605461 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:05.605704 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 15:03:05.605887 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
W20260808 15:03:05.609582 11656 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:05.608196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57408 (local address 127.2.180.193:39721)
0808 15:03:05.608327 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.608331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.608649 (+ 318us) server_negotiation.cc:407] Connection header received
0808 15:03:05.608755 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.608758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.608818 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.608886 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.609385 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57408: BlockingRecv error: recv got EOF from 127.0.0.1:57408 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 15:03:05.609818 2771 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786201338129629-2771-0/krb5kdc/krb5cc' of type FILE
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:05.611155 2771 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:05.612340 2771 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:03:05.625073 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.613535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57420 (local address 127.2.180.193:39721)
0808 15:03:05.613719 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.613725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.613866 (+ 141us) server_negotiation.cc:407] Connection header received
0808 15:03:05.613963 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.613966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.614012 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.614078 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.614859 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.615502 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.616666 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.617055 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.617137 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.620469 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:05.620473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:05.620476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:05.620502 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:05.622752 (+ 2250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.623363 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.623366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.623369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.623506 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.623854 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.623858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.623860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.624096 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:05.624385 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:05.624617 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 15:03:05.624818 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:03:05.626660 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 11055
I20260808 15:03:05.637053 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.627356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57426 (local address 127.2.180.193:39721)
0808 15:03:05.627584 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.627592 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.627830 (+ 238us) server_negotiation.cc:407] Connection header received
0808 15:03:05.627967 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.627970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.628033 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.628106 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.628901 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.629534 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.630591 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.630986 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.631082 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.632867 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:05.632871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:05.632874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:05.632901 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:05.635104 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.635622 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.635625 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.635627 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.635747 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.636050 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.636053 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.636055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.636273 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:05.636452 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:05.636673 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 15:03:05.636811 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:03:05.637514 11232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:05.718029 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 11055
I20260808 15:03:05.738952 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 11248
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:03:05.753871 11677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.739875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33848 (local address 127.2.180.194:33643)
0808 15:03:05.740414 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.740421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.740446 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:03:05.740508 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.740515 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.740738 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.740864 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.741779 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.742781 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.743969 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.744528 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.744651 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.748700 (+ 4049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:05.748711 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:05.748725 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:05.748766 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:05.751527 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.752163 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.752170 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.752172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.752297 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.752563 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.752566 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.752568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.752887 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:05.753117 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:05.753349 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 15:03:05.753541 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":259,"threads_started":1}
I20260808 15:03:05.755064 11443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:05.835410 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 11248
I20260808 15:03:05.856755 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 11456
Aug 08 15:03:05 dist-test-slave-jlzn krb5kdc[10899](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:03:05.871387 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:05.857560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41250 (local address 127.2.180.195:46407)
0808 15:03:05.858164 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:05.858171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:05.858200 (+ 29us) server_negotiation.cc:407] Connection header received
0808 15:03:05.858265 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:05.858271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:05.858474 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:05.858626 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:05.859672 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.860758 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.861996 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:05.862559 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:05.862677 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:05.866554 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:05.866565 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:05.866579 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:05.866612 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:05.869224 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.869782 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.869789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.869791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.869904 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:05.870178 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:05.870181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:05.870183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:05.870502 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:05.870723 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:05.870910 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 15:03:05.871100 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":272,"threads_started":1}
I20260808 15:03:05.872843 11633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:05.958307 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 11456
I20260808 15:03:05.979576 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 10917
I20260808 15:03:05.980726 11011 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:06.080039 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 10917
2026-08-08T15:03:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3991 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:03:06 dist-test-slave-jlzn krb5kdc[11686](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:03:06 dist-test-slave-jlzn krb5kdc[11686](info): set up 2 sockets
Aug 08 15:03:06 dist-test-slave-jlzn krb5kdc[11686](info): commencing operation
krb5kdc: starting...
W20260808 15:03:07.246416 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.004s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:03:07 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.254.keytab.
Entry for principal oryx/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.254.keytab.
I20260808 15:03:07.436455 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40807
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.254.keytab
--principal=oryx/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40807
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:07.746304 11709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:07.746584 11709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:07.746724 11709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:07.753762 11709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:07.753873 11709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:07.753952 11709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:07.754037 11709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:07.754096 11709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:07.763513 11709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.254.keytab
--principal=oryx/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:07.764770 11709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:07.766057 11709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:07.777241 11719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:07.777279 11723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:07.777323 11720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:07.778157 11709 server_base.cc:1034] running on GCE node
I20260808 15:03:07.778880 11709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:07.780448 11709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:07.781643 11709 hybrid_clock.cc:648] HybridClock initialized: now 1786201387781597 us; error 37 us; skew 500 ppm
I20260808 15:03:07.781955 11709 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:07 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:07.789166 11709 init.cc:377] Logged in from keytab as oryx/127.2.180.254@KRBTEST.COM (short username oryx)
I20260808 15:03:07.790504 11709 webserver.cc:460] Webserver started at http://127.2.180.254:44877/ using document root <none> and password file <none>
I20260808 15:03:07.791095 11709 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:07.791195 11709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:07.791435 11709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:07.794232 11709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "0bd40e8c8ed245cd862e7f502de9ae74"
format_stamp: "Formatted at 2026-08-08 15:03:07 on dist-test-slave-jlzn"
server_key: "e61a0676b59487aa220193fd160c58de"
server_key_iv: "179776aece000c495996958d8f49de35"
server_key_version: "encryptionkey@0"
I20260808 15:03:07.794916 11709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "0bd40e8c8ed245cd862e7f502de9ae74"
format_stamp: "Formatted at 2026-08-08 15:03:07 on dist-test-slave-jlzn"
server_key: "e61a0676b59487aa220193fd160c58de"
server_key_iv: "179776aece000c495996958d8f49de35"
server_key_version: "encryptionkey@0"
I20260808 15:03:07.799578 11709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 15:03:07.802791 11733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:07.804049 11709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:07.804185 11709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "0bd40e8c8ed245cd862e7f502de9ae74"
format_stamp: "Formatted at 2026-08-08 15:03:07 on dist-test-slave-jlzn"
server_key: "e61a0676b59487aa220193fd160c58de"
server_key_iv: "179776aece000c495996958d8f49de35"
server_key_version: "encryptionkey@0"
I20260808 15:03:07.804334 11709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:07.826901 11709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:07.830499 11709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:07.830770 11709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:07.847391 11709 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:40807
I20260808 15:03:07.847431 11811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:40807 every 8 connection(s)
I20260808 15:03:07.848630 11709 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:07.851333 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 11709
I20260808 15:03:07.851526 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:07.852226 2771 external_mini_cluster.cc:1428] Setting key cc302c5c9fbead80082bb9d73c2672f4
I20260808 15:03:07.852275 11812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:07.866055 11812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: Bootstrap starting.
Aug 08 15:03:07 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:07.869056 11812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:07.870052 11812 log.cc:824] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:07.872661 11812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: No bootstrap required, opened a new log
I20260808 15:03:07.873915 11816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:07.854045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38866 (local address 127.2.180.254:40807)
0808 15:03:07.854868 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:07.854878 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:07.854921 (+ 43us) server_negotiation.cc:407] Connection header received
0808 15:03:07.856687 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:07.856708 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:07.857063 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:07.857359 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:07.859163 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:07.860557 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:07.862195 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:07.862872 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:07.863002 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:07.867419 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:07.867426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:07.867438 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:07.867475 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:07.870290 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:07.870904 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:07.870910 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:07.870915 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:07.871088 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:07.871431 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:07.871436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:07.871438 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:07.871970 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:07.872253 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:07.872668 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 15:03:07.873031 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":342,"threads_started":1}
I20260808 15:03:07.879984 11812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } }
I20260808 15:03:07.880312 11812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:07.880399 11812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bd40e8c8ed245cd862e7f502de9ae74, State: Initialized, Role: FOLLOWER
I20260808 15:03:07.880909 11812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } }
I20260808 15:03:07.881134 11812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:07.881254 11812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:07.881409 11812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:07.883668 11812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } }
I20260808 15:03:07.884176 11812 leader_election.cc:302] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bd40e8c8ed245cd862e7f502de9ae74; no voters:
I20260808 15:03:07.884676 11812 leader_election.cc:288] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:07.884794 11819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:07.885718 11819 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 1 LEADER]: Becoming Leader. State: Replica: 0bd40e8c8ed245cd862e7f502de9ae74, State: Running, Role: LEADER
I20260808 15:03:07.886236 11819 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } }
I20260808 15:03:07.886632 11812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:07.888695 11823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } }
I20260808 15:03:07.889011 11823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:07.889683 11826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd40e8c8ed245cd862e7f502de9ae74. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } }
I20260808 15:03:07.890115 11826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:07.890725 11835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:07.896077 11835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:07.903723 11835 catalog_manager.cc:1347] Generated new cluster ID: c4bb6cc0d5124193a8a0698d89d499cd
I20260808 15:03:07.903832 11835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:07.936285 11835 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:07.937673 11835 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:07.949616 11835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: Generated new TSK 0
I20260808 15:03:07.950518 11835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab.
Entry for principal oryx/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab.
I20260808 15:03:08.013018 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40807
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab
--principal=oryx/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:08.325762 11857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:08.326013 11857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:08.326140 11857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:08.333029 11857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:08.333118 11857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:08.333232 11857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:08.342362 11857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab
--principal=oryx/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:08.343400 11857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:08.344604 11857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:08.356544 11866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:08.356590 11870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:08.356619 11867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:08.357575 11857 server_base.cc:1034] running on GCE node
I20260808 15:03:08.358139 11857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:08.358961 11857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:08.360164 11857 hybrid_clock.cc:648] HybridClock initialized: now 1786201388360097 us; error 51 us; skew 500 ppm
I20260808 15:03:08.360450 11857 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:08 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201388, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:08.363453 11857 init.cc:377] Logged in from keytab as oryx/127.2.180.193@KRBTEST.COM (short username oryx)
I20260808 15:03:08.364733 11857 webserver.cc:460] Webserver started at http://127.2.180.193:35651/ using document root <none> and password file <none>
I20260808 15:03:08.365357 11857 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:08.365470 11857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:08.365751 11857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:08.368463 11857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "a0a62b1e0d7f4ba78262b35859df9bb4"
format_stamp: "Formatted at 2026-08-08 15:03:08 on dist-test-slave-jlzn"
server_key: "20a12e1bc3c790382ec4c97e3d74cdf9"
server_key_iv: "4579965fcda7050d63f94a893ada7660"
server_key_version: "encryptionkey@0"
I20260808 15:03:08.369182 11857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "a0a62b1e0d7f4ba78262b35859df9bb4"
format_stamp: "Formatted at 2026-08-08 15:03:08 on dist-test-slave-jlzn"
server_key: "20a12e1bc3c790382ec4c97e3d74cdf9"
server_key_iv: "4579965fcda7050d63f94a893ada7660"
server_key_version: "encryptionkey@0"
I20260808 15:03:08.373993 11857 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 15:03:08.377174 11882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:08.378357 11857 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:03:08.378499 11857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "a0a62b1e0d7f4ba78262b35859df9bb4"
format_stamp: "Formatted at 2026-08-08 15:03:08 on dist-test-slave-jlzn"
server_key: "20a12e1bc3c790382ec4c97e3d74cdf9"
server_key_iv: "4579965fcda7050d63f94a893ada7660"
server_key_version: "encryptionkey@0"
I20260808 15:03:08.378662 11857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:08.400353 11857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:08.404172 11857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:08.404413 11857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:08.405279 11857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:08.406489 11857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:08.406565 11857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:08.406677 11857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:08.406725 11857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:08.434779 11857 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:44641
I20260808 15:03:08.434837 12065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:44641 every 8 connection(s)
I20260808 15:03:08.435951 11857 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:08.440079 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 11857
I20260808 15:03:08.440295 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:03:08.441088 2771 external_mini_cluster.cc:1428] Setting key 0a8b0431e9edba1204eee354175ee7d3
Aug 08 15:03:08 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201388, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:08.456116 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:08.438223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:53751 (local address 127.2.180.254:40807)
0808 15:03:08.438642 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:08.438646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:08.439388 (+ 742us) server_negotiation.cc:407] Connection header received
0808 15:03:08.441812 (+ 2424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:08.441818 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:08.441911 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:08.442033 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:08.443768 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:08.444820 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:08.446288 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:08.446800 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:08.446916 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:08.450340 (+ 3424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:08.450345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:08.450348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:08.450380 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:08.453246 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:08.453910 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:08.453916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:08.453920 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:08.454073 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:08.454427 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:08.454432 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:08.454435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:08.454804 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:08.454958 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:08.455510 (+ 552us) server_negotiation.cc:299] Negotiation successful
0808 15:03:08.455735 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":223,"threads_started":1}
I20260808 15:03:08.456761 12070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:08.438578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.193:53751)
0808 15:03:08.439233 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:08.439275 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:08.441519 (+ 2244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:08.442268 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:08.442279 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:08.442983 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:08.443587 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:08.443611 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:08.444951 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:08.444955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:08.446146 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:08.446158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:08.446974 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:08.446978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:08.447162 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:08.447752 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:08.447772 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:08.450118 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:08.453385 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:08.453391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:08.453401 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:08.453800 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:08.454164 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:08.454167 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:08.454169 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:08.454332 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:08.454913 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:08.454918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:08.455188 (+ 270us) client_negotiation.cc:769] Sending connection context
0808 15:03:08.455392 (+ 204us) client_negotiation.cc:240] Negotiation successful
0808 15:03:08.455735 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":246,"threads_started":1}
I20260808 15:03:08.458130 12067 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40807
I20260808 15:03:08.458424 12067 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:08.459268 12067 heartbeater.cc:511] Master 127.2.180.254:40807 requested a full tablet report, sending...
I20260808 15:03:08.461474 11759 ts_manager.cc:194] Registered new tserver with Master: a0a62b1e0d7f4ba78262b35859df9bb4 (127.2.180.193:44641)
I20260808 15:03:08.462823 11759 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:53751
WARNING: no policy specified for oryx/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab.
Entry for principal oryx/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab.
I20260808 15:03:08.488803 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40807
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab
--principal=oryx/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:08.796080 12074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:08.796350 12074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:08.796487 12074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:08.803309 12074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:08.803421 12074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:08.803571 12074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:08.812772 12074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab
--principal=oryx/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:08.813889 12074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:08.815088 12074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:08.827315 12081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:08.827487 12083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:08.827741 12080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:08.828073 12074 server_base.cc:1034] running on GCE node
I20260808 15:03:08.828639 12074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:08.829453 12074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:08.830654 12074 hybrid_clock.cc:648] HybridClock initialized: now 1786201388830591 us; error 57 us; skew 500 ppm
I20260808 15:03:08.830929 12074 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:08 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201388, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:08.833889 12074 init.cc:377] Logged in from keytab as oryx/127.2.180.194@KRBTEST.COM (short username oryx)
I20260808 15:03:08.835151 12074 webserver.cc:460] Webserver started at http://127.2.180.194:40597/ using document root <none> and password file <none>
I20260808 15:03:08.835798 12074 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:08.835947 12074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:08.836231 12074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:08.838963 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "fe501c99f632443e9ed883a3a33178e4"
format_stamp: "Formatted at 2026-08-08 15:03:08 on dist-test-slave-jlzn"
server_key: "6cc813c45ae81453dea69d95bf9c59aa"
server_key_iv: "82f0935c36d92c12dcd23da81133a149"
server_key_version: "encryptionkey@0"
I20260808 15:03:08.839689 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "fe501c99f632443e9ed883a3a33178e4"
format_stamp: "Formatted at 2026-08-08 15:03:08 on dist-test-slave-jlzn"
server_key: "6cc813c45ae81453dea69d95bf9c59aa"
server_key_iv: "82f0935c36d92c12dcd23da81133a149"
server_key_version: "encryptionkey@0"
I20260808 15:03:08.844381 12074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 15:03:08.847611 12093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:08.849133 12074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:03:08.849285 12074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "fe501c99f632443e9ed883a3a33178e4"
format_stamp: "Formatted at 2026-08-08 15:03:08 on dist-test-slave-jlzn"
server_key: "6cc813c45ae81453dea69d95bf9c59aa"
server_key_iv: "82f0935c36d92c12dcd23da81133a149"
server_key_version: "encryptionkey@0"
I20260808 15:03:08.849484 12074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:08.863591 12074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:08.868510 12074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:08.868813 12074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:08.869818 12074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:08.871490 12074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:08.871604 12074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:08.871722 12074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:08.871775 12074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:08.899824 12074 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:41537
I20260808 15:03:08.899893 12269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:41537 every 8 connection(s)
I20260808 15:03:08.900969 12074 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:08.904006 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 12074
I20260808 15:03:08.904202 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:03:08.904809 2771 external_mini_cluster.cc:1428] Setting key 46e239ee70c23e79f48cb7bf95b67380
Aug 08 15:03:08 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201388, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:08.919977 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:08.903282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:36871 (local address 127.2.180.254:40807)
0808 15:03:08.903451 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:08.903455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:08.904655 (+ 1200us) server_negotiation.cc:407] Connection header received
0808 15:03:08.906525 (+ 1870us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:08.906529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:08.906602 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:08.906692 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:08.908127 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:08.909205 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:08.910742 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:08.911264 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:08.911378 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:08.914680 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:08.914687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:08.914692 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:08.914728 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:08.917088 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:08.917912 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:08.917915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:08.917918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:08.918035 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:08.918441 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:08.918444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:08.918446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:08.918684 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:08.918795 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:08.919533 (+ 738us) server_negotiation.cc:299] Negotiation successful
0808 15:03:08.919691 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:03:08.921294 12272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:08.903705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.194:36871)
0808 15:03:08.904497 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:08.904543 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:08.906275 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:08.906883 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:08.906891 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:08.907375 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:08.907965 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:08.907984 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:08.909351 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:08.909355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:08.910608 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:08.910620 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:08.911442 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:08.911448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:08.911659 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:08.912294 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:08.912315 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:08.914459 (+ 2144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:08.917220 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:08.917229 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:08.917243 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:08.917815 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:08.918128 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:08.918133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:08.918137 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:08.918353 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:08.918810 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:08.918818 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:08.919199 (+ 381us) client_negotiation.cc:769] Sending connection context
0808 15:03:08.919465 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 15:03:08.919949 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":316,"threads_started":1}
I20260808 15:03:08.922631 12270 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40807
I20260808 15:03:08.922900 12270 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:08.923544 12270 heartbeater.cc:511] Master 127.2.180.254:40807 requested a full tablet report, sending...
I20260808 15:03:08.924969 11759 ts_manager.cc:194] Registered new tserver with Master: fe501c99f632443e9ed883a3a33178e4 (127.2.180.194:41537)
I20260808 15:03:08.925730 11759 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:36871
WARNING: no policy specified for oryx/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab.
Entry for principal oryx/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab.
I20260808 15:03:08.950531 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40807
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab
--principal=oryx/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:09.258329 12276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:09.258584 12276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:09.258713 12276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:09.265600 12276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:09.265698 12276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:09.265842 12276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:09.274919 12276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab
--principal=oryx/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:09.276067 12276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:09.277283 12276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:09.289630 12284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:09.289582 12285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:09.289662 12292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:09.290467 12276 server_base.cc:1034] running on GCE node
I20260808 15:03:09.291030 12276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:09.291968 12276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:09.293169 12276 hybrid_clock.cc:648] HybridClock initialized: now 1786201389293101 us; error 55 us; skew 500 ppm
I20260808 15:03:09.293452 12276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:09 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201389, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:09.296455 12276 init.cc:377] Logged in from keytab as oryx/127.2.180.195@KRBTEST.COM (short username oryx)
I20260808 15:03:09.297787 12276 webserver.cc:460] Webserver started at http://127.2.180.195:38623/ using document root <none> and password file <none>
I20260808 15:03:09.298430 12276 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:09.298543 12276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:09.298818 12276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:09.301599 12276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "3b2d76632a5c4aff888f5c3462ef9172"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "a2450367729aff7293603b19258aa8a2"
server_key_iv: "9724e3a0140f8907f180308d3f8a3ac8"
server_key_version: "encryptionkey@0"
I20260808 15:03:09.302320 12276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "3b2d76632a5c4aff888f5c3462ef9172"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "a2450367729aff7293603b19258aa8a2"
server_key_iv: "9724e3a0140f8907f180308d3f8a3ac8"
server_key_version: "encryptionkey@0"
I20260808 15:03:09.307163 12276 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:03:09.310477 12301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:09.311692 12276 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:03:09.311858 12276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "3b2d76632a5c4aff888f5c3462ef9172"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "a2450367729aff7293603b19258aa8a2"
server_key_iv: "9724e3a0140f8907f180308d3f8a3ac8"
server_key_version: "encryptionkey@0"
I20260808 15:03:09.312031 12276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:09.338019 12276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:09.341596 12276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:09.341848 12276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:09.342729 12276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:09.343964 12276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:09.344041 12276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:09.344166 12276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:09.344221 12276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:09.373520 12276 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:33331
I20260808 15:03:09.373572 12462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:33331 every 8 connection(s)
I20260808 15:03:09.374665 12276 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:09.376024 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 12276
I20260808 15:03:09.376186 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:03:09.376602 2771 external_mini_cluster.cc:1428] Setting key 886f294d58b0d558b94a11330fa08288
Aug 08 15:03:09 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201389, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:09.393036 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.377010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:33955 (local address 127.2.180.254:40807)
0808 15:03:09.377209 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:09.377216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:09.378478 (+ 1262us) server_negotiation.cc:407] Connection header received
0808 15:03:09.380338 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:09.380342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:09.380406 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:09.380484 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:09.381858 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.382617 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.384174 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.384565 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.384653 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.387745 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:09.387749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:09.387753 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:09.387783 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:09.390367 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.391053 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.391057 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.391059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.391213 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.391632 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.391635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.391637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.391918 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:09.392038 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:09.392618 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 15:03:09.392794 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:03:09.393867 12465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.377490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.195:33955)
0808 15:03:09.378320 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:09.378364 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:09.380110 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:09.380647 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:09.380654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:09.381117 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:09.381703 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.381723 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.382752 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.382756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.384052 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.384076 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.384725 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.384730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.384911 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.385497 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:09.385519 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:09.387574 (+ 2055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:09.390487 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.390495 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.390508 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.390950 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.391326 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.391331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.391335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.391534 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.392020 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:09.392025 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:09.392293 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 15:03:09.392538 (+ 245us) client_negotiation.cc:240] Negotiation successful
0808 15:03:09.392914 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":330,"threads_started":1}
I20260808 15:03:09.394958 12463 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40807
I20260808 15:03:09.395227 12463 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:09.395902 12463 heartbeater.cc:511] Master 127.2.180.254:40807 requested a full tablet report, sending...
I20260808 15:03:09.397168 11759 ts_manager.cc:194] Registered new tserver with Master: 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331)
I20260808 15:03:09.397943 11759 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:33955
I20260808 15:03:09.403141 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.180.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.253.keytab.
Entry for principal oryx/127.2.180.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.253.keytab.
I20260808 15:03:09.450734 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.253:37237
--webserver_interface=127.2.180.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.253.keytab
--principal=oryx/127.2.180.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40807,127.2.180.253:37237
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:03:09.465866 12067 heartbeater.cc:503] Master 127.2.180.254:40807 was elected leader, sending a full tablet report...
W20260808 15:03:09.764811 12469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:09.765091 12469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:09.765218 12469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:09.772101 12469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:09.772207 12469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:09.772279 12469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:09.772353 12469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:09.772416 12469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:09.781873 12469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40807,127.2.180.253:37237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.253.keytab
--principal=oryx/127.2.180.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.253:37237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.180.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
W20260808 15:03:09.782205 12469 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.2.180.254:40807,127.2.180.253:37237'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 15:03:09.783241 12469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:09.784746 12469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:09.796069 12476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:09.796036 12482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:09.795997 12478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:09.796744 12469 server_base.cc:1034] running on GCE node
I20260808 15:03:09.797439 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:09.798645 12469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:09.799876 12469 hybrid_clock.cc:648] HybridClock initialized: now 1786201389799803 us; error 48 us; skew 500 ppm
I20260808 15:03:09.800163 12469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:09 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201389, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:09.803041 12469 init.cc:377] Logged in from keytab as oryx/127.2.180.253@KRBTEST.COM (short username oryx)
I20260808 15:03:09.804361 12469 webserver.cc:460] Webserver started at http://127.2.180.253:45845/ using document root <none> and password file <none>
I20260808 15:03:09.804992 12469 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:09.805094 12469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:09.805378 12469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:09.808265 12469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data/instance:
uuid: "7b67415abc55416c87a38d8fe934b740"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "2401f9460cb861e56abd843cef39cf41"
server_key_iv: "e50cafef5f570395cd50edc4a0eac3af"
server_key_version: "encryptionkey@0"
I20260808 15:03:09.809062 12469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal/instance:
uuid: "7b67415abc55416c87a38d8fe934b740"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "2401f9460cb861e56abd843cef39cf41"
server_key_iv: "e50cafef5f570395cd50edc4a0eac3af"
server_key_version: "encryptionkey@0"
I20260808 15:03:09.815187 12469 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260808 15:03:09.818526 12490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:09.819758 12469 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:03:09.819941 12469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
uuid: "7b67415abc55416c87a38d8fe934b740"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "2401f9460cb861e56abd843cef39cf41"
server_key_iv: "e50cafef5f570395cd50edc4a0eac3af"
server_key_version: "encryptionkey@0"
I20260808 15:03:09.820130 12469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:09.852799 12469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:09.856618 12469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:09.856841 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:09.873526 12469 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.253:37237
I20260808 15:03:09.873566 12560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.253:37237 every 8 connection(s)
I20260808 15:03:09.874760 12469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
I20260808 15:03:09.876169 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 12469
I20260808 15:03:09.878616 12561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:09.887111 12561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:03:09 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.253@KRBTEST.COM
Aug 08 15:03:09 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201389, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.253@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:09.900074 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.878252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.2.180.253:37237)
0808 15:03:09.879047 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:09.879071 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:09.879120 (+ 49us) server_negotiation.cc:407] Connection header received
0808 15:03:09.881275 (+ 2155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:09.881303 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:09.881821 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:09.882267 (+ 446us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:09.883633 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.885601 (+ 1968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.886879 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.887950 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.888130 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.892434 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:09.892444 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:09.892457 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:09.892498 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:09.895679 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.896462 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.896468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.896473 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.896655 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.897057 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.897068 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.897070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.897604 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:09.897880 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:09.898370 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 15:03:09.898740 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":344,"threads_started":1}
I20260808 15:03:09.904476 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.889799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38876 (local address 127.2.180.254:40807)
0808 15:03:09.889946 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:09.889950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:09.890613 (+ 663us) server_negotiation.cc:407] Connection header received
0808 15:03:09.890771 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:09.890775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:09.890839 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:09.890920 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:09.892076 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.893168 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.894840 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.895457 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.895601 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.898680 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:09.898687 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:09.898692 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:09.898731 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:09.901806 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.902500 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.902505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.902509 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.902684 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.903137 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.903143 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.903146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.903474 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:09.903589 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:09.904122 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 15:03:09.904272 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:03:09.904726 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.889850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:38876)
0808 15:03:09.890477 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:09.890517 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:09.890655 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:09.890998 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:09.891004 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:09.891611 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:09.891932 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.891943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.893324 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.893328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.894697 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.894710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.895621 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.895625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.895750 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.896312 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:09.896333 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:09.898511 (+ 2178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:09.901925 (+ 3414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.901936 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.901942 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.902389 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.902818 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.902823 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.902826 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.902979 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.903597 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:09.903604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:09.903794 (+ 190us) client_negotiation.cc:769] Sending connection context
0808 15:03:09.904057 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 15:03:09.904344 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":327,"threads_started":1}
I20260808 15:03:09.906093 12561 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:03:09 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201389, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.253@KRBTEST.COM for oryx/127.2.180.253@KRBTEST.COM
I20260808 15:03:09.916647 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.906617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.0.0.1:42444)
0808 15:03:09.906761 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:09.906776 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:09.906844 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:09.907113 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:09.907115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:09.907394 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:09.907517 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.907526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.908501 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.908504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.909362 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.909374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.909991 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.909995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.910131 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.910519 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:09.910533 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:09.912345 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:09.914707 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.914710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.914712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.915090 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.915403 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.915406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.915408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.915529 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.915996 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:09.915999 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:09.916130 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 15:03:09.916251 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 15:03:09.916435 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 15:03:09.916899 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.906797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42444 (local address 127.2.180.253:37237)
0808 15:03:09.906927 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:09.906931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:09.906944 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:03:09.906978 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:09.906981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:09.907026 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:09.907096 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:09.907649 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.908378 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.909474 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.909876 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.909963 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.912490 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:09.912498 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:09.912501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:09.912528 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:09.914596 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.915179 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.915183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.915185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.915315 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.915612 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.915615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.915617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.915900 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:09.916002 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:09.916344 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 15:03:09.916566 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":45}
I20260808 15:03:09.920890 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Bootstrap starting.
I20260808 15:03:09.923743 12561 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:09.924789 12561 log.cc:824] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:09.927146 12561 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: No bootstrap required, opened a new log
I20260808 15:03:09.927960 12270 heartbeater.cc:503] Master 127.2.180.254:40807 was elected leader, sending a full tablet report...
I20260808 15:03:09.933974 12561 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } }
I20260808 15:03:09.934250 12561 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:09.934343 12561 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b67415abc55416c87a38d8fe934b740, State: Initialized, Role: FOLLOWER
I20260808 15:03:09.934837 12561 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } }
I20260808 15:03:09.935559 12567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } } }
I20260808 15:03:09.935892 12567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:03:09.936450 12561 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 15:03:09.944303 12585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:03:09.944410 12585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:03:09.946669 12469 master_runner.cc:428] Detected that this master 7b67415abc55416c87a38d8fe934b740 is joining an existing cluster
I20260808 15:03:09.946748 12469 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.180.253:37237
I20260808 15:03:09.968050 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.950560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38888 (local address 127.2.180.254:40807)
0808 15:03:09.950730 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:09.950735 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:09.952468 (+ 1733us) server_negotiation.cc:407] Connection header received
0808 15:03:09.952654 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:09.952659 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:09.952739 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:09.952845 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:09.955006 (+ 2161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.956232 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.958250 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.958941 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.959052 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.961598 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:09.961605 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:09.961609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:09.961648 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:09.964983 (+ 3335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.965739 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.965745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.965748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.965925 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.966356 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.966362 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.966365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.966749 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:09.966898 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:09.967436 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 15:03:09.967677 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:03:09.968089 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.950628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:38888)
0808 15:03:09.952346 (+ 1718us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:09.952370 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:09.952523 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:09.952872 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:09.952877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:09.953541 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:09.954829 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.954844 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.956430 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.956436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.958063 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.958080 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.959348 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.959357 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.959537 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.960143 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:09.960174 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:09.961384 (+ 1210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:09.965126 (+ 3742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.965130 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.965134 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.965604 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.966044 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.966048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.966052 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.966236 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.966888 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:09.966894 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:09.967088 (+ 194us) client_negotiation.cc:769] Sending connection context
0808 15:03:09.967290 (+ 202us) client_negotiation.cc:240] Negotiation successful
0808 15:03:09.967594 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1572,"thread_start_us":1422,"threads_started":1}
I20260808 15:03:09.975371 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.951252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42450 (local address 127.2.180.253:37237)
0808 15:03:09.951449 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:09.951455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:09.959697 (+ 8242us) server_negotiation.cc:407] Connection header received
0808 15:03:09.960346 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:09.960351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:09.960419 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:09.960477 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:09.961819 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.963072 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.965129 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.965806 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.965867 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.968046 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:09.968053 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:09.968057 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:09.968093 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:09.971347 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.972106 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.972111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.972114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.972269 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.972768 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.972772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.972776 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.974164 (+ 1388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:09.974300 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:09.974789 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 15:03:09.975004 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:03:09.976598 12597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.952204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.0.0.1:42450)
0808 15:03:09.959591 (+ 7387us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:09.959614 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:09.960221 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:09.960664 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:09.960668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:09.961349 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:09.961687 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.961701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.963474 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.963489 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.965009 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.965023 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.966110 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.966116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.966272 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.966805 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:09.966830 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:09.967882 (+ 1052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:09.971453 (+ 3571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.971458 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.971465 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.971999 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.972386 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.972390 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.972393 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.972679 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.974361 (+ 1682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:09.974365 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:09.974525 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 15:03:09.975875 (+ 1350us) client_negotiation.cc:240] Negotiation successful
0808 15:03:09.976162 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7260,"thread_start_us":7490,"threads_started":1}
Aug 08 15:03:09 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.254@KRBTEST.COM for oryx/127.2.180.253@KRBTEST.COM
WARNING: no policy specified for oryx/127.2.180.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.252@KRBTEST.COM" created.
I20260808 15:03:09.996592 12564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.974414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42452 (local address 127.2.180.253:37237)
0808 15:03:09.975840 (+ 1426us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:09.975845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:09.980737 (+ 4892us) server_negotiation.cc:407] Connection header received
0808 15:03:09.981075 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:09.981080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:09.981155 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:09.981247 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:09.982344 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.983308 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.984962 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.985635 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.985767 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.989678 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:09.989685 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:09.989690 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:09.989725 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:09.993319 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.994179 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.994185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.994188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.994358 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:09.994840 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:09.994846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:09.994849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:09.995204 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:09.995333 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:09.996000 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 15:03:09.996217 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1300,"thread_start_us":5588,"threads_started":1}
I20260808 15:03:09.997741 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:09.974396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.0.0.1:42452)
0808 15:03:09.980614 (+ 6218us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:09.980662 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:09.980859 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:09.981341 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:09.981349 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:09.982004 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:09.982199 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.982207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.983421 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.983425 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.984790 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:09.984806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:09.985815 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:09.985822 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:09.986022 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:09.986674 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:09.986705 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:09.989502 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:09.993470 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.993482 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.993489 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.994032 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.994531 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:09.994536 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:09.994540 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:09.994699 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:09.995430 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:09.995439 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:09.995655 (+ 216us) client_negotiation.cc:769] Sending connection context
0808 15:03:09.996986 (+ 1331us) client_negotiation.cc:240] Negotiation successful
0808 15:03:09.997315 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6015,"thread_start_us":5885,"threads_started":1}
I20260808 15:03:09.999938 11759 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.180.253:37237: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7b67415abc55416c87a38d8fe934b740"
member_type: NON_VOTER
last_known_addr {
host: "127.2.180.253"
port: 37237
}
attrs {
promote: true
}
}
dest_uuid: "0bd40e8c8ed245cd862e7f502de9ae74"
cas_config_opid_index: -1
I20260808 15:03:10.001055 11759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } }
I20260808 15:03:10.014634 12608 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7b67415abc55416c87a38d8fe934b740 (127.2.180.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } }
I20260808 15:03:10.016137 12526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:10.019366 12608 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.180.253:37237
I20260808 15:03:10.020031 12609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } }
I20260808 15:03:10.020296 12469 master.cc:537] Master@127.2.180.253:37237 shutting down...
I20260808 15:03:10.020320 12609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.252.keytab.
Entry for principal oryx/127.2.180.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.252.keytab.
I20260808 15:03:10.021997 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.252:44587
--webserver_interface=127.2.180.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.252.keytab
--principal=oryx/127.2.180.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40807,127.2.180.253:37237,127.2.180.252:44587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:03:10.020153 12526 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Refusing update from remote peer 0bd40e8c8ed245cd862e7f502de9ae74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 15:03:10.030575 12609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:03:10.033649 11737 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 15:03:10.035213 11737 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 -> Peer 7b67415abc55416c87a38d8fe934b740 (127.2.180.253:37237): Couldn't send request to peer 7b67415abc55416c87a38d8fe934b740. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 15:03:10.035365 12469 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:10.035724 12469 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:10.035917 12469 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: stopping tablet replica
I20260808 15:03:10.042037 12469 master.cc:552] Master@127.2.180.253:37237 shutdown complete.
I20260808 15:03:10.042164 12469 master_runner.cc:305] Clearing existing system tablet
I20260808 15:03:10.042557 12469 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:10.042654 12469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:10.044848 12469 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 15:03:10.046216 12617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:10.046532 12469 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:10.046773 12469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:03:10.046859 12469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
uuid: "7b67415abc55416c87a38d8fe934b740"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "2401f9460cb861e56abd843cef39cf41"
server_key_iv: "e50cafef5f570395cd50edc4a0eac3af"
server_key_version: "encryptionkey@0"
I20260808 15:03:10.050297 12469 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:10.054873 12469 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 15:03:10.055027 12469 log.cc:1196] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 15:03:10.055428 12469 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Deleting consensus metadata
I20260808 15:03:10.055657 12469 master_runner.cc:315] Copying system tablet from 127.2.180.254:40807
I20260808 15:03:10.060634 12469 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: tablet copy: Beginning tablet copy session from remote peer at address 127.2.180.254:40807
I20260808 15:03:10.074723 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.061284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38892 (local address 127.2.180.254:40807)
0808 15:03:10.061472 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.061479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.064296 (+ 2817us) server_negotiation.cc:407] Connection header received
0808 15:03:10.064472 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.064478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.064557 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.064641 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.065477 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.066177 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.067379 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.067784 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.067939 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.069646 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.069651 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.069654 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.069686 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.072261 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.072844 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.072848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.072850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.072977 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.073336 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.073341 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.073343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.073625 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.073755 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.074171 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.074380 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:03:10.074767 12624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.061234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:38892)
0808 15:03:10.064195 (+ 2961us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.064212 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.064344 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.064685 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.064689 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.065122 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.065330 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.065339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.066319 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.066325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.067231 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.067243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.067946 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.067952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.068122 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.068683 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.068700 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.069389 (+ 689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.072382 (+ 2993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.072388 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.072392 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.072734 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.073088 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.073093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.073097 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.073233 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.073740 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.073744 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.073911 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.074039 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.074325 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2836,"thread_start_us":2729,"threads_started":1}
I20260808 15:03:10.075671 11791 tablet_copy_service.cc:140] P 0bd40e8c8ed245cd862e7f502de9ae74: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7b67415abc55416c87a38d8fe934b740 ({username='oryx', principal='oryx/127.2.180.253@KRBTEST.COM'} at 127.0.0.1:38892)
I20260808 15:03:10.075948 11791 tablet_copy_service.cc:161] P 0bd40e8c8ed245cd862e7f502de9ae74: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7b67415abc55416c87a38d8fe934b740 at {username='oryx', principal='oryx/127.2.180.253@KRBTEST.COM'} at 127.0.0.1:38892: session id = 7b67415abc55416c87a38d8fe934b740-00000000000000000000000000000000
I20260808 15:03:10.078928 11791 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 15:03:10.080698 12469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 15:03:10.085278 12469 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: tablet copy: Starting download of 0 data blocks...
I20260808 15:03:10.085538 12469 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: tablet copy: Starting download of 1 WAL segments...
I20260808 15:03:10.087890 12469 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 15:03:10.090835 11791 tablet_copy_service.cc:342] P 0bd40e8c8ed245cd862e7f502de9ae74: Request end of tablet copy session 7b67415abc55416c87a38d8fe934b740-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.180.253@KRBTEST.COM'} at 127.0.0.1:38892
I20260808 15:03:10.090989 11791 tablet_copy_service.cc:433] P 0bd40e8c8ed245cd862e7f502de9ae74: ending tablet copy session 7b67415abc55416c87a38d8fe934b740-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7b67415abc55416c87a38d8fe934b740
I20260808 15:03:10.100415 12469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:10.102787 12625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:10.103951 12626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:10.115513 12629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:10.115690 12469 server_base.cc:1034] running on GCE node
I20260808 15:03:10.116166 12469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:10.116719 12469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:10.117918 12469 hybrid_clock.cc:648] HybridClock initialized: now 1786201390117844 us; error 49 us; skew 500 ppm
I20260808 15:03:10.118121 12469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201390, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:10.119822 12469 init.cc:377] Logged in from keytab as oryx/127.2.180.253@KRBTEST.COM (short username oryx)
I20260808 15:03:10.120998 12469 webserver.cc:460] Webserver started at http://127.2.180.253:39925/ using document root <none> and password file <none>
I20260808 15:03:10.121287 12469 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:10.121361 12469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:10.123584 12469 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260808 15:03:10.125180 12643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:10.125550 12469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 15:03:10.125691 12469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
uuid: "7b67415abc55416c87a38d8fe934b740"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "2401f9460cb861e56abd843cef39cf41"
server_key_iv: "e50cafef5f570395cd50edc4a0eac3af"
server_key_version: "encryptionkey@0"
I20260808 15:03:10.125831 12469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:10.142742 12469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:10.145865 12469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:10.182085 12469 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.253:37237
I20260808 15:03:10.182106 12716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.253:37237 every 8 connection(s)
I20260808 15:03:10.183077 12469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
I20260808 15:03:10.188835 12717 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:03:10.193305 12717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Bootstrap starting.
I20260808 15:03:10.209249 12717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:03:10.209669 12717 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: Bootstrap complete.
I20260808 15:03:10.210461 12717 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } }
I20260808 15:03:10.210686 12717 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7b67415abc55416c87a38d8fe934b740, State: Initialized, Role: LEARNER
I20260808 15:03:10.210901 12717 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } }
I20260808 15:03:10.211720 12720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } }
I20260808 15:03:10.212066 12717 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:10.212181 12720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:03:10.220536 12734 catalog_manager.cc:1259] Loaded cluster ID: c4bb6cc0d5124193a8a0698d89d499cd
I20260808 15:03:10.220613 12734 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: loading cluster ID for follower catalog manager: success
I20260808 15:03:10.222939 12734 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: acquiring CA information for follower catalog manager: success
I20260808 15:03:10.224794 12734 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:03:10.400063 12463 heartbeater.cc:503] Master 127.2.180.254:40807 was elected leader, sending a full tablet report...
W20260808 15:03:10.405114 12613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:10.405381 12613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:10.405508 12613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:10.412614 12613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:10.412724 12613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:10.412801 12613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:10.412878 12613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:10.412935 12613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:10.422500 12613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40807,127.2.180.253:37237,127.2.180.252:44587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.252.keytab
--principal=oryx/127.2.180.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.252:44587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.180.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:10.423692 12613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:10.425349 12613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:03:10.431052 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.424114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.0.0.1:42468)
0808 15:03:10.424373 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.424393 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.424515 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.425508 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.425514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.425546 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:10.425708 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.425716 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.427531 (+ 1815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.427535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.428841 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.428855 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.430373 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.430377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.430528 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.430567 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.430646 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.430787 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 15:03:10.431136 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.424300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.2.180.253:37237)
0808 15:03:10.425118 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.425124 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.425143 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:03:10.425220 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.425224 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.425308 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.425440 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:10.425910 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.427393 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.429000 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.430237 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.430373 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.430602 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.430683 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.430820 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":507,"threads_started":1}
I20260808 15:03:10.432260 12686 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 15:03:10.432857 12720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd40e8c8ed245cd862e7f502de9ae74. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } }
I20260808 15:03:10.433139 12720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: This master's current role is: LEARNER
W20260808 15:03:10.438647 12745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:10.438841 12749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:10.439208 12746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:10.439445 12613 server_base.cc:1034] running on GCE node
I20260808 15:03:10.440212 12613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:10.441447 12613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:10.442652 12613 hybrid_clock.cc:648] HybridClock initialized: now 1786201390442607 us; error 24 us; skew 500 ppm
I20260808 15:03:10.442965 12613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201390, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:10.445935 12613 init.cc:377] Logged in from keytab as oryx/127.2.180.252@KRBTEST.COM (short username oryx)
I20260808 15:03:10.447196 12613 webserver.cc:460] Webserver started at http://127.2.180.252:41515/ using document root <none> and password file <none>
I20260808 15:03:10.447825 12613 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:10.447950 12613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:10.448226 12613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:10.450953 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data/instance:
uuid: "216720daa1104240aa8aa7ba1c268acd"
format_stamp: "Formatted at 2026-08-08 15:03:10 on dist-test-slave-jlzn"
server_key: "69a4ef3dbb0f9d472c17a58518920f2c"
server_key_iv: "3bcb0fb8072887c7f84f2dd8dede8a5d"
server_key_version: "encryptionkey@0"
I20260808 15:03:10.451758 12613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal/instance:
uuid: "216720daa1104240aa8aa7ba1c268acd"
format_stamp: "Formatted at 2026-08-08 15:03:10 on dist-test-slave-jlzn"
server_key: "69a4ef3dbb0f9d472c17a58518920f2c"
server_key_iv: "3bcb0fb8072887c7f84f2dd8dede8a5d"
server_key_version: "encryptionkey@0"
I20260808 15:03:10.456627 12613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:03:10.459681 12759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:10.460891 12613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:03:10.461043 12613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
uuid: "216720daa1104240aa8aa7ba1c268acd"
format_stamp: "Formatted at 2026-08-08 15:03:10 on dist-test-slave-jlzn"
server_key: "69a4ef3dbb0f9d472c17a58518920f2c"
server_key_iv: "3bcb0fb8072887c7f84f2dd8dede8a5d"
server_key_version: "encryptionkey@0"
I20260808 15:03:10.461205 12613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:10.483693 12613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:10.487402 12613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:10.487629 12613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:10.503320 12613 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.252:44587
I20260808 15:03:10.503351 12848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.252:44587 every 8 connection(s)
I20260808 15:03:10.504539 12613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
I20260808 15:03:10.506028 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 12613
I20260808 15:03:10.508536 12850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:10.517295 12850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.252@KRBTEST.COM
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201390, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.252@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:10.530025 12857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.508017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58614 (local address 127.2.180.252:44587)
0808 15:03:10.508781 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.508797 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.508851 (+ 54us) server_negotiation.cc:407] Connection header received
0808 15:03:10.511100 (+ 2249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.511127 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.511599 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.512047 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.513417 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.515476 (+ 2059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.516941 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.517969 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.518147 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.522779 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.522786 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.522795 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.522832 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.526120 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.526819 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.526825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.526830 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.527002 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.527388 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.527393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.527395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.528112 (+ 717us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.528410 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.528834 (+ 424us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.529165 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":305,"threads_started":1}
I20260808 15:03:10.534118 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.520061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38896 (local address 127.2.180.254:40807)
0808 15:03:10.520252 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.520258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.520717 (+ 459us) server_negotiation.cc:407] Connection header received
0808 15:03:10.520878 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.520883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.520951 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.521034 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.522248 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.522974 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.524984 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.525449 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.525543 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.528777 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.528782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.528785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.528817 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.531513 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.532254 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.532257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.532260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.532396 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.532754 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.532758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.532760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.533025 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.533161 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.533597 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.533800 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:03:10.534142 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.519905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:38896)
0808 15:03:10.520577 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.520610 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.520762 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.521113 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.521120 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.521779 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.522092 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.522106 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.523101 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.523106 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.524847 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.524863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.525641 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.525648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.525835 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.526422 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.526443 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.528605 (+ 2162us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.531627 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.531638 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.531645 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.532136 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.532501 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.532506 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.532510 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.532639 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.533146 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.533153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.533310 (+ 157us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.533462 (+ 152us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.533756 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":365,"threads_started":1}
I20260808 15:03:10.535446 12850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201390, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.252@KRBTEST.COM for oryx/127.2.180.253@KRBTEST.COM
I20260808 15:03:10.545790 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.536108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.0.0.1:42474)
0808 15:03:10.536277 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.536291 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.536361 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.536589 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.536591 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.536864 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.536981 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.536988 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.537865 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.537868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.538749 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.538760 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.539353 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.539357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.539464 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.539907 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.539923 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.541581 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.544079 (+ 2498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.544082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.544085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.544423 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.544702 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.544705 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.544707 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.544819 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.545208 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.545211 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.545313 (+ 102us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.545405 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.545581 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 15:03:10.545857 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.536143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42474 (local address 127.2.180.253:37237)
0808 15:03:10.536339 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.536344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.536361 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:03:10.536438 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.536441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.536498 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.536569 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.537080 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.537768 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.538848 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.539250 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.539336 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.541750 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.541754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.541758 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.541788 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.543981 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.544508 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.544511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.544513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.544630 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.544890 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.544892 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.544894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.545130 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.545233 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.545472 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.545634 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:03:10.546692 12850 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201390, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.252@KRBTEST.COM for oryx/127.2.180.252@KRBTEST.COM
I20260808 15:03:10.557277 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.547179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44587 (local address 127.0.0.1:58622)
0808 15:03:10.547347 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.547363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.547444 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.547692 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.547697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.548030 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.548164 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.548172 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.549020 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.549024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.549799 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.549809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.550436 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.550442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.550598 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.551031 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.551047 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.552742 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.555180 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.555185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.555189 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.555508 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.555852 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.555857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.555860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.555996 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.556447 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.556452 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.556574 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.556681 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.556905 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260808 15:03:10.557422 12857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.547246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.2.180.252:44587)
0808 15:03:10.547378 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.547383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.547400 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:03:10.547530 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.547535 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.547587 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.547656 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.548280 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.548910 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.549915 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.550301 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.550398 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.552892 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.552897 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.552900 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.552926 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.555064 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.555607 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.555612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.555616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.555731 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.556089 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.556094 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.556098 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.556344 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.556460 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.556782 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.557056 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":44}
I20260808 15:03:10.561553 12850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Bootstrap starting.
I20260808 15:03:10.564625 12850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:10.565660 12850 log.cc:824] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:10.568188 12850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: No bootstrap required, opened a new log
I20260808 15:03:10.574854 12850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } }
I20260808 15:03:10.575131 12850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:10.575218 12850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 216720daa1104240aa8aa7ba1c268acd, State: Initialized, Role: FOLLOWER
I20260808 15:03:10.575703 12850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } }
I20260808 15:03:10.576481 12860 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } } }
I20260808 15:03:10.576836 12860 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:03:10.577358 12850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:10.584403 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 11857
W20260808 15:03:10.585893 12877 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:03:10.586028 12877 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:03:10.590083 12613 master_runner.cc:428] Detected that this master 216720daa1104240aa8aa7ba1c268acd is joining an existing cluster
I20260808 15:03:10.590204 12613 master_runner.cc:432] Initiating AddMaster RPC to add 127.2.180.252:44587
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.193@KRBTEST.COM
I20260808 15:03:10.607136 12880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.585580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.2.180.193:44641)
0808 15:03:10.586571 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.586578 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.586600 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:03:10.586662 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.586667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.586854 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.588796 (+ 1942us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.589039 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.590517 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.592754 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.593542 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.593707 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.599766 (+ 6059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.599780 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.599795 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.599865 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.603599 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.604452 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.604461 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.604465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.604625 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.605090 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.605095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.605098 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.605529 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.605827 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.606451 (+ 624us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.606716 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":681,"threads_started":1}
I20260808 15:03:10.609473 12059 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:10.628999 12888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.594309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:38908)
0808 15:03:10.609492 (+ 15183us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.609513 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.609919 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.610469 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.610475 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.611250 (+ 775us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.611669 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.611683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.613210 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.613216 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.615059 (+ 1843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.615076 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.616416 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.616423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.616661 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.617518 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.617543 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.618779 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.625759 (+ 6980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.625764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.625768 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.626303 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.626826 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.626831 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.626835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.627025 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.627744 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.627749 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.628005 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.628141 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.628436 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15039,"mutex_wait_us":75,"spinlock_wait_cycles":768,"thread_start_us":15350,"threads_started":1}
I20260808 15:03:10.629441 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.596168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.2.180.253:37237)
0808 15:03:10.596343 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.596349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.609913 (+ 13564us) server_negotiation.cc:407] Connection header received
0808 15:03:10.610165 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.610171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.610252 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.610317 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.611950 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.613102 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.615334 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.616023 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.616132 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.618786 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.618793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.618798 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.618836 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.622461 (+ 3625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.626652 (+ 4191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.626658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.626661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.626826 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.627477 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.627482 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.627486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.627841 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.627984 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.628590 (+ 606us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.628837 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:03:10.629643 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.594721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38908 (local address 127.2.180.254:40807)
0808 15:03:10.594901 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.594907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.609605 (+ 14698us) server_negotiation.cc:407] Connection header received
0808 15:03:10.610091 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.610097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.610184 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.610261 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.611887 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.613044 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.615307 (+ 2263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.615970 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.616095 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.619029 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.619036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.619041 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.619083 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.625645 (+ 6562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.626460 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.626465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.626468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.626624 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.627168 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.627173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.627176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.627533 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.627668 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.629091 (+ 1423us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.629304 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:03:10.630790 12886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.596404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.0.0.1:42480)
0808 15:03:10.609798 (+ 13394us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.609821 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.610017 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.610526 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.610531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.611340 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.611749 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.611762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.613334 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.613340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.615174 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.615189 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.616428 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.616435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.616679 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.617378 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.617400 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.618576 (+ 1176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.625849 (+ 7273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.625854 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.625858 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.626509 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.626981 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.626985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.626988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.627262 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.628070 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.628075 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.628297 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.630005 (+ 1708us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.630335 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13216,"mutex_wait_us":70,"spinlock_wait_cycles":22400,"thread_start_us":12784,"threads_started":1}
I20260808 15:03:10.636804 12857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.609714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.2.180.252:44587)
0808 15:03:10.609883 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.609888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.620194 (+ 10306us) server_negotiation.cc:407] Connection header received
0808 15:03:10.620313 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.620317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.620388 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.620468 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.621770 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.622877 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.624915 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.625596 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.625693 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.628181 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.628188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.628193 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.628226 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.632996 (+ 4770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.633772 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.633778 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.633781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.634031 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.634397 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.634402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.634405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.634713 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.634865 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.636249 (+ 1384us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.636451 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:03:10.638983 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.609982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44587 (local address 127.0.0.1:58636)
0808 15:03:10.620095 (+ 10113us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.620108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.620211 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.620614 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.620619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.621264 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.621632 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.621643 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.623176 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.623182 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.624769 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.624785 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.625980 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.625987 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.626142 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.626843 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.626864 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.628005 (+ 1141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.633165 (+ 5160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.633170 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.633174 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.633653 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.634137 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.634141 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.634145 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.634290 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.634927 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.634931 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.635109 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.638324 (+ 3215us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.638568 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10015,"thread_start_us":9862,"threads_started":1}
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.254@KRBTEST.COM for oryx/127.2.180.252@KRBTEST.COM
I20260808 15:03:10.652890 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.635948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44587 (local address 127.0.0.1:58638)
0808 15:03:10.636201 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.636219 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.636317 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.638206 (+ 1889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.638212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.638644 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.638835 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.638846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.640557 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.640563 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.641920 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.641936 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.642952 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.642959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.643114 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.643676 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.643697 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.646317 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.650250 (+ 3933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.650255 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.650259 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.650733 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.651108 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.651113 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.651117 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.651274 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.651923 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.651928 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.652082 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.652248 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.652521 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 15:03:10.652918 12857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.635988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58638 (local address 127.2.180.252:44587)
0808 15:03:10.637602 (+ 1614us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.637607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.637623 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:03:10.637714 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.637720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.637793 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.638167 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.639013 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.640381 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.642096 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.642783 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.642901 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.646527 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.646534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.646539 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.646575 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.650103 (+ 3528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.650853 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.650859 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.650863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.650997 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.651391 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.651396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.651399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.651759 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.651944 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.652378 (+ 434us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.652585 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1520,"thread_start_us":2018,"threads_started":1}
I20260808 15:03:10.654738 11759 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.2.180.252:44587: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "216720daa1104240aa8aa7ba1c268acd"
member_type: NON_VOTER
last_known_addr {
host: "127.2.180.252"
port: 44587
}
attrs {
promote: true
}
}
dest_uuid: "0bd40e8c8ed245cd862e7f502de9ae74"
cas_config_opid_index: 5
I20260808 15:03:10.655735 11759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } }
I20260808 15:03:10.657358 12609 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 216720daa1104240aa8aa7ba1c268acd (127.2.180.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.657985 12686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 LEARNER]: Refusing update from remote peer 0bd40e8c8ed245cd862e7f502de9ae74: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 15:03:10.659225 12798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:10.661262 12609 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.2.180.252:44587
I20260808 15:03:10.670003 12609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 15:03:10.670687 12893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.671118 12893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:10.680361 12798 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Refusing update from remote peer 0bd40e8c8ed245cd862e7f502de9ae74: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 15:03:10.680836 12613 master.cc:537] Master@127.2.180.252:44587 shutting down...
I20260808 15:03:10.681872 12686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 216720daa1104240aa8aa7ba1c268acd (127.2.180.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.683862 12609 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:10.689776 12720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: NON_VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: true } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.690040 12720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: This master's current role is: LEARNER
I20260808 15:03:10.691510 12609 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: attempting to promote NON_VOTER 7b67415abc55416c87a38d8fe934b740 to VOTER
I20260808 15:03:10.692576 12609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } }
I20260808 15:03:10.694624 12613 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:10.694989 12613 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:10.695179 12613 tablet_replica.cc:330] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: stopping tablet replica
I20260808 15:03:10.695493 12686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 LEARNER]: Refusing update from remote peer 0bd40e8c8ed245cd862e7f502de9ae74: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 15:03:10.696292 12894 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260808 15:03:10.697965 11737 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 -> Peer 216720daa1104240aa8aa7ba1c268acd (127.2.180.252:44587): Couldn't send request to peer 216720daa1104240aa8aa7ba1c268acd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 15:03:10.699061 12893 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7b67415abc55416c87a38d8fe934b740 (127.2.180.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.701951 12613 master.cc:552] Master@127.2.180.252:44587 shutdown complete.
I20260808 15:03:10.702082 12613 master_runner.cc:305] Clearing existing system tablet
I20260808 15:03:10.702515 12613 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:10.702697 12613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:10.706418 12686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7b67415abc55416c87a38d8fe934b740 (127.2.180.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.710374 12720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.710615 12720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:03:10.711290 12609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.711596 12609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:10.711880 12609 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.712118 12609 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:10.712944 12613 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.000s sys 0.009s
I20260808 15:03:10.714314 12904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:10.714677 12613 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:10.714957 12613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 15:03:10.715075 12613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
uuid: "216720daa1104240aa8aa7ba1c268acd"
format_stamp: "Formatted at 2026-08-08 15:03:10 on dist-test-slave-jlzn"
server_key: "69a4ef3dbb0f9d472c17a58518920f2c"
server_key_iv: "3bcb0fb8072887c7f84f2dd8dede8a5d"
server_key_version: "encryptionkey@0"
I20260808 15:03:10.718712 12613 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:10.723107 12613 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 15:03:10.723248 12613 log.cc:1196] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 15:03:10.723623 12613 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Deleting consensus metadata
I20260808 15:03:10.723940 12613 master_runner.cc:315] Copying system tablet from 127.2.180.254:40807
I20260808 15:03:10.726171 12613 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: tablet copy: Beginning tablet copy session from remote peer at address 127.2.180.254:40807
I20260808 15:03:10.737929 12069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.726814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38914 (local address 127.2.180.254:40807)
0808 15:03:10.727076 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:10.727082 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:10.727501 (+ 419us) server_negotiation.cc:407] Connection header received
0808 15:03:10.727743 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:10.727749 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:10.727848 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:10.727938 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:10.728819 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.729563 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.730841 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.731246 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.731353 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.733032 (+ 1679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:10.733037 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:10.733040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:10.733070 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:10.735439 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.736087 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.736091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.736093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.736218 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:10.736568 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:10.736572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:10.736574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:10.736845 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:10.736983 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:10.737361 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 15:03:10.737583 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260808 15:03:10.737955 12915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:10.726800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:38914)
0808 15:03:10.727405 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:10.727425 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:10.727621 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:10.727992 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:10.727997 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:10.728487 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:10.728681 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.728690 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.729699 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.729708 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.730702 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:10.730717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:10.731379 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:10.731385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:10.731588 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:10.732156 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:10.732174 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:10.732847 (+ 673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:10.735560 (+ 2713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.735565 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.735570 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.735974 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.736320 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:10.736325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:10.736329 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:10.736471 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:10.736957 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:10.736962 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:10.737121 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 15:03:10.737242 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 15:03:10.737520 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":403,"threads_started":1}
I20260808 15:03:10.738646 11791 tablet_copy_service.cc:140] P 0bd40e8c8ed245cd862e7f502de9ae74: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 216720daa1104240aa8aa7ba1c268acd ({username='oryx', principal='oryx/127.2.180.252@KRBTEST.COM'} at 127.0.0.1:38914)
I20260808 15:03:10.738792 11791 tablet_copy_service.cc:161] P 0bd40e8c8ed245cd862e7f502de9ae74: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 216720daa1104240aa8aa7ba1c268acd at {username='oryx', principal='oryx/127.2.180.252@KRBTEST.COM'} at 127.0.0.1:38914: session id = 216720daa1104240aa8aa7ba1c268acd-00000000000000000000000000000000
I20260808 15:03:10.741066 11791 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 15:03:10.742413 12613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 15:03:10.746889 12613 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: tablet copy: Starting download of 0 data blocks...
I20260808 15:03:10.747145 12613 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: tablet copy: Starting download of 1 WAL segments...
I20260808 15:03:10.748991 12613 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 15:03:10.751822 11791 tablet_copy_service.cc:342] P 0bd40e8c8ed245cd862e7f502de9ae74: Request end of tablet copy session 216720daa1104240aa8aa7ba1c268acd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.2.180.252@KRBTEST.COM'} at 127.0.0.1:38914
I20260808 15:03:10.751940 11791 tablet_copy_service.cc:433] P 0bd40e8c8ed245cd862e7f502de9ae74: ending tablet copy session 216720daa1104240aa8aa7ba1c268acd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 216720daa1104240aa8aa7ba1c268acd
I20260808 15:03:10.762131 12613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:10.764468 12916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:10.770749 12917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:10.773788 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 11857
I20260808 15:03:10.785831 12613 server_base.cc:1034] running on GCE node
W20260808 15:03:10.786449 12919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:10.786972 12613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:10.787621 12613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:10.790926 12613 hybrid_clock.cc:648] HybridClock initialized: now 1786201390790889 us; error 62 us; skew 500 ppm
I20260808 15:03:10.791198 12613 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:10 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201390, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:10.793339 12613 init.cc:377] Logged in from keytab as oryx/127.2.180.252@KRBTEST.COM (short username oryx)
I20260808 15:03:10.799162 12613 webserver.cc:460] Webserver started at http://127.2.180.252:33747/ using document root <none> and password file <none>
I20260808 15:03:10.799479 12613 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:10.799582 12613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:10.802798 12613 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:03:10.804684 12931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:10.805157 12613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260808 15:03:10.805325 12613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
uuid: "216720daa1104240aa8aa7ba1c268acd"
format_stamp: "Formatted at 2026-08-08 15:03:10 on dist-test-slave-jlzn"
server_key: "69a4ef3dbb0f9d472c17a58518920f2c"
server_key_iv: "3bcb0fb8072887c7f84f2dd8dede8a5d"
server_key_version: "encryptionkey@0"
I20260808 15:03:10.805583 12613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:10.806835 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:44641
--local_ip_for_outbound_sockets=127.2.180.193
--tserver_master_addrs=127.2.180.254:40807
--webserver_port=35651
--webserver_interface=127.2.180.193
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab
--principal=oryx/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237,127.2.180.252:44587 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:03:10.823994 12613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:10.827931 12613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:10.846402 12613 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.252:44587
I20260808 15:03:10.846459 13019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.252:44587 every 8 connection(s)
I20260808 15:03:10.847406 12613 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
I20260808 15:03:10.864995 13021 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:03:10.885075 13021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Bootstrap starting.
I20260808 15:03:10.911733 13021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:03:10.912299 13021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Bootstrap complete.
I20260808 15:03:10.913386 13021 raft_consensus.cc:348] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } }
I20260808 15:03:10.913686 13021 raft_consensus.cc:729] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 216720daa1104240aa8aa7ba1c268acd, State: Initialized, Role: LEARNER
I20260808 15:03:10.913959 13021 consensus_queue.cc:260] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } }
I20260808 15:03:10.914773 13025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:10.915266 13025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: This master's current role is: LEARNER
I20260808 15:03:10.917443 13021 sys_catalog.cc:564] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:10.939015 13042 catalog_manager.cc:1259] Loaded cluster ID: c4bb6cc0d5124193a8a0698d89d499cd
I20260808 15:03:10.939118 13042 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: loading cluster ID for follower catalog manager: success
I20260808 15:03:10.942508 13042 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: acquiring CA information for follower catalog manager: success
I20260808 15:03:10.944901 13042 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:03:11.120297 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.113608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58648 (local address 127.2.180.252:44587)
0808 15:03:11.114372 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.114376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.114392 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:03:11.114454 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.114458 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.114531 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.114632 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:11.115131 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.116297 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.118489 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.119333 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.119446 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.119631 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.119932 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.120035 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":528,"threads_started":1}
I20260808 15:03:11.120332 12601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.113517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44587 (local address 127.0.0.1:58648)
0808 15:03:11.113713 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:11.113736 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:11.113888 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:11.114726 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:11.114733 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:11.114770 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:11.114971 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.114984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.116418 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.116423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.118343 (+ 1920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.118358 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.119512 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.119518 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.119749 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.119786 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 15:03:11.119901 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 15:03:11.120052 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 15:03:11.121428 12984 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 15:03:11.122018 13025 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bd40e8c8ed245cd862e7f502de9ae74. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:11.122308 13025 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: This master's current role is: LEARNER
W20260808 15:03:11.190294 12938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:11.190572 12938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:11.190711 12938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:11.197844 12938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:11.197949 12938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:11.198096 12938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:11.211113 12938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab
--principal=oryx/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:44641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=35651
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237,127.2.180.252:44587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:11.212913 12938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:11.214639 12938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:11.230046 13051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:11.230057 13053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:11.230973 13056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:11.231462 12938 server_base.cc:1034] running on GCE node
I20260808 15:03:11.232184 12938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:11.233202 12938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:11.234416 12938 hybrid_clock.cc:648] HybridClock initialized: now 1786201391234354 us; error 50 us; skew 500 ppm
I20260808 15:03:11.234762 12938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201391, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:11.238610 12938 init.cc:377] Logged in from keytab as oryx/127.2.180.193@KRBTEST.COM (short username oryx)
I20260808 15:03:11.239984 12938 webserver.cc:460] Webserver started at http://127.2.180.193:35651/ using document root <none> and password file <none>
I20260808 15:03:11.240762 12938 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:11.240906 12938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:11.247529 12938 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:03:11.250857 13070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:11.252485 12938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:03:11.252652 12938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "a0a62b1e0d7f4ba78262b35859df9bb4"
format_stamp: "Formatted at 2026-08-08 15:03:08 on dist-test-slave-jlzn"
server_key: "20a12e1bc3c790382ec4c97e3d74cdf9"
server_key_iv: "4579965fcda7050d63f94a893ada7660"
server_key_version: "encryptionkey@0"
I20260808 15:03:11.253566 12938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:11.285386 12938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:11.289386 12938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:11.289628 12938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:11.290621 12938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:11.291977 12938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:11.292073 12938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:11.292183 12938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:11.292255 12938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:11.324121 12938 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:44641
I20260808 15:03:11.324246 13241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:44641 every 8 connection(s)
I20260808 15:03:11.325531 12938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:11.328738 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 12938
I20260808 15:03:11.329109 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 12074
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201391, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201391, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.253@KRBTEST.COM
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201391, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.252@KRBTEST.COM
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.194@KRBTEST.COM
I20260808 15:03:11.358377 13247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.328167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:34113 (local address 127.2.180.253:37237)
0808 15:03:11.340626 (+ 12459us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.340632 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.340662 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:03:11.340716 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.340721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.340802 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.341904 (+ 1102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:11.342071 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.343115 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.346599 (+ 3484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.347188 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.347957 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.350867 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:11.350874 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:11.350879 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:11.350917 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:11.354667 (+ 3750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.355479 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.355484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.355487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.355651 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.356200 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.356205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.356208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.356574 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:11.357186 (+ 612us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.357690 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.357978 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12327,"thread_start_us":12716,"threads_started":1}
I20260808 15:03:11.359524 13248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.328901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.2.180.193:34113)
0808 15:03:11.329873 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:11.329930 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:11.332720 (+ 2790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:11.341035 (+ 8315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:11.341041 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:11.341491 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:11.341767 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.341777 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.343255 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.343260 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.346426 (+ 3166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.346440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.347421 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.347427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.347627 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.348232 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:11.348253 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:11.350638 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:11.354810 (+ 4172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.354815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.354819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.355323 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.355779 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.355784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.355788 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.356087 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.356730 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:11.356777 (+ 47us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:11.357252 (+ 475us) client_negotiation.cc:769] Sending connection context
0808 15:03:11.357607 (+ 355us) client_negotiation.cc:240] Negotiation successful
0808 15:03:11.358132 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"mutex_wait_us":20,"spinlock_wait_cycles":19712,"thread_start_us":571,"threads_started":1}
I20260808 15:03:11.360746 13252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.329409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.193:54291)
0808 15:03:11.334703 (+ 5294us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:11.334722 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:11.334846 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:11.335756 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:11.335768 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:11.336740 (+ 972us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:11.337639 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.337663 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.339311 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.339317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.341416 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.341429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.342578 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.342585 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.342822 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.343662 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:11.343691 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:11.347495 (+ 3804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:11.354203 (+ 6708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.354211 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.354235 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.354948 (+ 713us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.355473 (+ 525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.355477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.355480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.356021 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.356823 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:11.356828 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:11.359956 (+ 3128us) client_negotiation.cc:769] Sending connection context
0808 15:03:11.360092 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 15:03:11.360341 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5161,"mutex_wait_us":32,"thread_start_us":16728,"threads_started":1}
I20260808 15:03:11.361151 13244 heartbeater.cc:347] Connected to a master server at 127.2.180.253:37237
I20260808 15:03:11.361443 13244 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:11.361548 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.331350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:54197 (local address 127.2.180.252:44587)
0808 15:03:11.331541 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.331547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.346537 (+ 14990us) server_negotiation.cc:407] Connection header received
0808 15:03:11.346667 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.346671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.346754 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.346843 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:11.348220 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.349273 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.350774 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.351169 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.351276 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.354983 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:11.354990 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:11.354995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:11.355036 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:11.358565 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.359319 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.359324 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.359328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.359460 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.359931 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.359936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.359939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.360291 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:11.360421 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.360983 (+ 562us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.361211 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:03:11.361622 13250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.329659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:54291 (local address 127.2.180.254:40807)
0808 15:03:11.334262 (+ 4603us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.334268 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.334948 (+ 680us) server_negotiation.cc:407] Connection header received
0808 15:03:11.335024 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.335030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.335129 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.335286 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:11.337866 (+ 2580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.339117 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.341577 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.342167 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.342258 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.347751 (+ 5493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:11.347757 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:11.347762 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:11.347823 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:11.353973 (+ 6150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.355119 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.355124 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.355127 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.355339 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.356175 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.356180 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.356184 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.356624 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:11.356759 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.361026 (+ 4267us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.361261 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4473,"thread_start_us":4368,"threads_started":1}
I20260808 15:03:11.362272 13244 heartbeater.cc:511] Master 127.2.180.253:37237 requested a full tablet report, sending...
I20260808 15:03:11.362519 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.330525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44587 (local address 127.2.180.193:54197)
0808 15:03:11.346434 (+ 15909us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:11.346451 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:11.346562 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:11.347018 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:11.347023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:11.347675 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:11.348026 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.348039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.349436 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.349440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.350632 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.350645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.351340 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.351346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.351489 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.352124 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:11.352146 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:11.354760 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:11.358829 (+ 4069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.358834 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.358837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.359197 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.359597 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.359602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.359605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.359770 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.360529 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:11.360533 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:11.360695 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 15:03:11.361814 (+ 1119us) client_negotiation.cc:240] Negotiation successful
0808 15:03:11.362093 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15791,"thread_start_us":3894,"threads_started":1}
I20260808 15:03:11.363528 13242 heartbeater.cc:347] Connected to a master server at 127.2.180.252:44587
I20260808 15:03:11.363669 13242 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:11.364084 13243 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40807
I20260808 15:03:11.364176 13242 heartbeater.cc:511] Master 127.2.180.252:44587 requested a full tablet report, sending...
I20260808 15:03:11.364217 13243 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:11.364650 13243 heartbeater.cc:511] Master 127.2.180.254:40807 requested a full tablet report, sending...
I20260808 15:03:11.364693 12672 ts_manager.cc:194] Registered new tserver with Master: a0a62b1e0d7f4ba78262b35859df9bb4 (127.2.180.193:44641)
I20260808 15:03:11.365229 13253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.330960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33278 (local address 127.2.180.194:41537)
0808 15:03:11.342360 (+ 11400us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.342369 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.342399 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:03:11.342470 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.342478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.342721 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.343920 (+ 1199us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:11.345175 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.346659 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.348452 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.349257 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.349946 (+ 689us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.355583 (+ 5637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:11.355597 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:11.355612 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:11.355669 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:11.359418 (+ 3749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.360727 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.360736 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.360739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.361043 (+ 304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.361482 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.361487 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.361490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.361915 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:11.362214 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.363418 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.363668 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11210,"thread_start_us":11884,"threads_started":1}
I20260808 15:03:11.365886 11759 ts_manager.cc:194] Re-registered known tserver with Master: a0a62b1e0d7f4ba78262b35859df9bb4 (127.2.180.193:44641)
I20260808 15:03:11.365885 12962 ts_manager.cc:194] Registered new tserver with Master: a0a62b1e0d7f4ba78262b35859df9bb4 (127.2.180.193:44641)
I20260808 15:03:11.366726 12263 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:11.366837 11759 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:54291
I20260808 15:03:11.446839 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 12074
I20260808 15:03:11.466161 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:41537
--local_ip_for_outbound_sockets=127.2.180.194
--tserver_master_addrs=127.2.180.254:40807
--webserver_port=40597
--webserver_interface=127.2.180.194
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab
--principal=oryx/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237,127.2.180.252:44587 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:03:11.679407 12894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: NON_VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: true } } }
I20260808 15:03:11.679785 12894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:11.681650 13258 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: attempting to promote NON_VOTER 216720daa1104240aa8aa7ba1c268acd to VOTER
I20260808 15:03:11.682404 13258 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } }
I20260808 15:03:11.684242 12686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Refusing update from remote peer 0bd40e8c8ed245cd862e7f502de9ae74: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 15:03:11.684286 12984 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 LEARNER]: Refusing update from remote peer 0bd40e8c8ed245cd862e7f502de9ae74: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 15:03:11.684803 13258 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 15:03:11.685171 13257 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 15:03:11.687700 12894 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 216720daa1104240aa8aa7ba1c268acd (127.2.180.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:11.690387 13258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:11.690696 13258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:11.690814 12686 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 216720daa1104240aa8aa7ba1c268acd (127.2.180.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:11.691174 12894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:11.691432 12894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:11.691676 12894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:11.691931 12894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:11.692070 12984 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 216720daa1104240aa8aa7ba1c268acd (127.2.180.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:11.696267 13263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:11.696640 13263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:03:11.697110 13265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:11.697399 13265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: This master's current role is: FOLLOWER
W20260808 15:03:11.787294 13256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:11.787576 13256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:11.787717 13256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:11.794854 13256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:11.794961 13256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:11.795102 13256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:11.804268 13256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab
--principal=oryx/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:41537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=40597
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237,127.2.180.252:44587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:11.805474 13256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:11.806747 13256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:11.819186 13272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:11.819223 13273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:11.819444 13276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:11.820327 13256 server_base.cc:1034] running on GCE node
I20260808 15:03:11.820837 13256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:11.821682 13256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:11.822881 13256 hybrid_clock.cc:648] HybridClock initialized: now 1786201391822825 us; error 46 us; skew 500 ppm
I20260808 15:03:11.823184 13256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201391, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:11.826267 13256 init.cc:377] Logged in from keytab as oryx/127.2.180.194@KRBTEST.COM (short username oryx)
I20260808 15:03:11.827633 13256 webserver.cc:460] Webserver started at http://127.2.180.194:40597/ using document root <none> and password file <none>
I20260808 15:03:11.828290 13256 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:11.828528 13256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:11.833583 13256 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:03:11.836143 13286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:11.837366 13256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:03:11.837518 13256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "fe501c99f632443e9ed883a3a33178e4"
format_stamp: "Formatted at 2026-08-08 15:03:08 on dist-test-slave-jlzn"
server_key: "6cc813c45ae81453dea69d95bf9c59aa"
server_key_iv: "82f0935c36d92c12dcd23da81133a149"
server_key_version: "encryptionkey@0"
I20260808 15:03:11.838418 13256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:11.865127 13256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:11.868690 13256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:11.868919 13256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:11.869827 13256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:11.871064 13256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:11.871158 13256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:11.871256 13256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:11.871325 13256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:11.900137 13256 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:41537
I20260808 15:03:11.900219 13449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:41537 every 8 connection(s)
I20260808 15:03:11.901407 13256 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:11.905264 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 13256
I20260808 15:03:11.905555 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 12276
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201391, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201391, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.252@KRBTEST.COM
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201391, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.253@KRBTEST.COM
I20260808 15:03:11.933548 13458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.903879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:57215 (local address 127.2.180.254:40807)
0808 15:03:11.904725 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.904732 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.906131 (+ 1399us) server_negotiation.cc:407] Connection header received
0808 15:03:11.909423 (+ 3292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.909429 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.909538 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.909614 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:11.912996 (+ 3382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.914123 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.920430 (+ 6307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.921113 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.921220 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.926153 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:11.926160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:11.926164 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:11.926214 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:11.929758 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.930670 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.930676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.930679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.930840 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.931428 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.931433 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.931436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.931834 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:11.931982 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.932995 (+ 1013us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.933226 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":615,"threads_started":1}
I20260808 15:03:11.933611 13457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.903839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:36747 (local address 127.2.180.252:44587)
0808 15:03:11.905083 (+ 1244us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.905090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.905965 (+ 875us) server_negotiation.cc:407] Connection header received
0808 15:03:11.909003 (+ 3038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.909010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.909098 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.909193 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:11.915280 (+ 6087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.916599 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.922286 (+ 5687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.922904 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.922999 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.927052 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:11.927059 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:11.927063 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:11.927105 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:11.930563 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.931281 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.931287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.931290 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.931436 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.931931 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.931936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.931939 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.932275 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:11.932404 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.932985 (+ 581us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.933218 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1111,"thread_start_us":1008,"threads_started":1}
Aug 08 15:03:11 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.195@KRBTEST.COM
I20260808 15:03:11.935624 13460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.904999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44587 (local address 127.2.180.194:36747)
0808 15:03:11.905817 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:11.905861 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:11.908569 (+ 2708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:11.910119 (+ 1550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:11.910129 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:11.910886 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:11.911637 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.911661 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.917782 (+ 6121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.917787 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.920267 (+ 2480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.920280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.923312 (+ 3032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.923318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.923532 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.924135 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:11.924170 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:11.926857 (+ 2687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:11.930698 (+ 3841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.930703 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.930707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.931130 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.931605 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.931610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.931613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.931779 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.932492 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:11.932497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:11.932694 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 15:03:11.933844 (+ 1150us) client_negotiation.cc:240] Negotiation successful
0808 15:03:11.934462 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"mutex_wait_us":134,"spinlock_wait_cycles":40832,"thread_start_us":977,"threads_started":1}
I20260808 15:03:11.935644 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.904521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.194:57215)
0808 15:03:11.905820 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:11.905861 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:11.908698 (+ 2837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:11.911901 (+ 3203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:11.911905 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:11.912455 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:11.912749 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.912761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.914345 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.914350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.919549 (+ 5199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.919565 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.921560 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.921567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.921859 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.922611 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:11.922644 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:11.925922 (+ 3278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:11.929917 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.929926 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.929941 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.930519 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.930999 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.931003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.931007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.931291 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.932114 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:11.932122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:11.932526 (+ 404us) client_negotiation.cc:769] Sending connection context
0808 15:03:11.933811 (+ 1285us) client_negotiation.cc:240] Negotiation successful
0808 15:03:11.934311 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"mutex_wait_us":82,"thread_start_us":613,"threads_started":1}
I20260808 15:03:11.937142 13450 heartbeater.cc:347] Connected to a master server at 127.2.180.252:44587
I20260808 15:03:11.937171 13452 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40807
I20260808 15:03:11.937477 13452 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:11.937484 13450 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:11.938372 13450 heartbeater.cc:511] Master 127.2.180.252:44587 requested a full tablet report, sending...
I20260808 15:03:11.938395 13452 heartbeater.cc:511] Master 127.2.180.254:40807 requested a full tablet report, sending...
I20260808 15:03:11.939971 11759 ts_manager.cc:194] Re-registered known tserver with Master: fe501c99f632443e9ed883a3a33178e4 (127.2.180.194:41537)
I20260808 15:03:11.940207 12962 ts_manager.cc:194] Registered new tserver with Master: fe501c99f632443e9ed883a3a33178e4 (127.2.180.194:41537)
I20260808 15:03:11.941082 11759 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:57215
I20260808 15:03:11.941855 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.906776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:38817 (local address 127.2.180.253:37237)
0808 15:03:11.919084 (+ 12308us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.919091 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.923061 (+ 3970us) server_negotiation.cc:407] Connection header received
0808 15:03:11.923450 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.923455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.923543 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.923676 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:11.925327 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.926526 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.929523 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.930137 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.930243 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.934833 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:11.934840 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:11.934845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:11.934891 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:11.938852 (+ 3961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.939614 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.939619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.939622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.939773 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.940250 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.940255 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.940257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.940586 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:11.940714 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.941287 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.941522 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12184,"thread_start_us":12077,"threads_started":1}
I20260808 15:03:11.942736 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.906357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.2.180.194:38817)
0808 15:03:11.922862 (+ 16505us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:11.922879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:11.923285 (+ 406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:11.924174 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:11.924178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:11.924798 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:11.925147 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.925160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.926800 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.926805 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.929358 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:11.929374 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.930568 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.930574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:11.930768 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.931290 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:11.931311 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:11.934629 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:11.939017 (+ 4388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.939021 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.939025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.939492 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.939946 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:11.939951 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:11.939954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:11.940110 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:11.940808 (+ 698us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:11.940813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:11.940975 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 15:03:11.942093 (+ 1118us) client_negotiation.cc:240] Negotiation successful
0808 15:03:11.942353 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16385,"thread_start_us":16732,"threads_started":1}
I20260808 15:03:11.943235 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:11.906789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.2.180.195:33331)
0808 15:03:11.922231 (+ 15442us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:11.922239 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:11.922266 (+ 27us) server_negotiation.cc:407] Connection header received
0808 15:03:11.922340 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:11.922348 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:11.922556 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:11.922723 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:11.924966 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.926460 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.927874 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:11.928699 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:11.928827 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:11.935486 (+ 6659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:11.935502 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:11.935517 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:11.935562 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:11.939457 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.940261 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.940268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.940271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.940422 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:11.940880 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:11.940884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:11.940887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:11.941289 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:11.941555 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:11.942598 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0808 15:03:11.942855 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15296,"thread_start_us":15136,"threads_started":1}
I20260808 15:03:11.944046 13453 heartbeater.cc:347] Connected to a master server at 127.2.180.253:37237
I20260808 15:03:11.944164 13453 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:11.944278 12458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:11.944571 13453 heartbeater.cc:511] Master 127.2.180.253:37237 requested a full tablet report, sending...
I20260808 15:03:11.945585 12672 ts_manager.cc:194] Registered new tserver with Master: fe501c99f632443e9ed883a3a33178e4 (127.2.180.194:41537)
I20260808 15:03:12.026558 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 12276
I20260808 15:03:12.047480 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:33331
--local_ip_for_outbound_sockets=127.2.180.195
--tserver_master_addrs=127.2.180.254:40807
--webserver_port=38623
--webserver_interface=127.2.180.195
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab
--principal=oryx/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237,127.2.180.252:44587 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:12.366230 13469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:12.366500 13469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:12.366631 13469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 15:03:12.369444 13243 heartbeater.cc:503] Master 127.2.180.254:40807 was elected leader, sending a full tablet report...
W20260808 15:03:12.374500 13469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:12.374600 13469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:12.374716 13469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:12.383775 13469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab
--principal=oryx/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:33331
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=38623
--tserver_master_addrs=127.2.180.254:40807,127.2.180.253:37237,127.2.180.252:44587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:12.384912 13469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:12.386089 13469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:12.398439 13480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:12.398429 13481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:12.398631 13484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:12.399246 13469 server_base.cc:1034] running on GCE node
I20260808 15:03:12.399887 13469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:12.400708 13469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:12.401903 13469 hybrid_clock.cc:648] HybridClock initialized: now 1786201392401855 us; error 35 us; skew 500 ppm
I20260808 15:03:12.402189 13469 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:12 dist-test-slave-jlzn krb5kdc[11686](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201392, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:12.405129 13469 init.cc:377] Logged in from keytab as oryx/127.2.180.195@KRBTEST.COM (short username oryx)
I20260808 15:03:12.406435 13469 webserver.cc:460] Webserver started at http://127.2.180.195:38623/ using document root <none> and password file <none>
I20260808 15:03:12.407143 13469 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:12.407281 13469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:12.412529 13469 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:12.415162 13493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:12.416513 13469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 15:03:12.416646 13469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "3b2d76632a5c4aff888f5c3462ef9172"
format_stamp: "Formatted at 2026-08-08 15:03:09 on dist-test-slave-jlzn"
server_key: "a2450367729aff7293603b19258aa8a2"
server_key_iv: "9724e3a0140f8907f180308d3f8a3ac8"
server_key_version: "encryptionkey@0"
I20260808 15:03:12.417622 13469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:12.439527 13469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:12.443387 13469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:12.443631 13469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:12.444559 13469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:12.445783 13469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:12.445890 13469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:12.445976 13469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:12.446030 13469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:12.475363 13469 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:33331
I20260808 15:03:12.475426 13651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:33331 every 8 connection(s)
I20260808 15:03:12.476967 13469 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:12.486789 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 13469
Aug 08 15:03:12 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201392, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.253@KRBTEST.COM
Aug 08 15:03:12 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201392, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:12.520938 13661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.480260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.2.180.195:50321)
0808 15:03:12.495693 (+ 15433us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:12.495710 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:12.495837 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:12.497028 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:12.497040 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:12.498000 (+ 960us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:12.498921 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.498946 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.503887 (+ 4941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.503893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.505462 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.505474 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.506821 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.506828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.507193 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.508094 (+ 901us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:12.508135 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:12.511233 (+ 3098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:12.515604 (+ 4371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:12.515612 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:12.515629 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:12.516267 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:12.516755 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:12.516760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:12.516764 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:12.516991 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:12.517794 (+ 803us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:12.517802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:12.518236 (+ 434us) client_negotiation.cc:769] Sending connection context
0808 15:03:12.518557 (+ 321us) client_negotiation.cc:240] Negotiation successful
0808 15:03:12.519072 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15274,"mutex_wait_us":71,"spinlock_wait_cycles":46848,"thread_start_us":14985,"threads_started":1}
I20260808 15:03:12.522598 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.479740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:50321 (local address 127.2.180.253:37237)
0808 15:03:12.496270 (+ 16530us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.496276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.496296 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:03:12.496359 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.496363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.496447 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.496586 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:12.500107 (+ 3521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.501334 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.505622 (+ 4288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.506420 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.506541 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.511485 (+ 4944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:12.511493 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:12.511509 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:12.511571 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:12.515400 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.516419 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.516425 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.516430 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.516622 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.517154 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.517160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.517164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.517545 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:12.517713 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.521909 (+ 4196us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.522162 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16396,"thread_start_us":16281,"threads_started":1}
Aug 08 15:03:12 dist-test-slave-jlzn krb5kdc[11686](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201392, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.252@KRBTEST.COM
I20260808 15:03:12.530069 13655 heartbeater.cc:347] Connected to a master server at 127.2.180.253:37237
I20260808 15:03:12.530407 13655 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:12.531401 13655 heartbeater.cc:511] Master 127.2.180.253:37237 requested a full tablet report, sending...
I20260808 15:03:12.533782 13666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.497333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44587 (local address 127.2.180.195:50455)
0808 15:03:12.505224 (+ 7891us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:12.505242 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:12.505356 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:12.511442 (+ 6086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:12.511447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:12.512247 (+ 800us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:12.512619 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.512631 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.514340 (+ 1709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.514346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.517153 (+ 2807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.517170 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.519039 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.519046 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.519257 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.520194 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:12.520215 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:12.524106 (+ 3891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:12.530834 (+ 6728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:12.530839 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:12.530845 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:12.531406 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:12.531892 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:12.531898 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:12.531902 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:12.532089 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:12.532789 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:12.532794 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:12.532953 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 15:03:12.533100 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:03:12.533365 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7777,"thread_start_us":7632,"threads_started":1}
I20260808 15:03:12.534308 12672 ts_manager.cc:194] Registered new tserver with Master: 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331)
I20260808 15:03:12.534647 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.479674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:43939 (local address 127.2.180.254:40807)
0808 15:03:12.496740 (+ 17066us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.496746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.496770 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:03:12.496838 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.496844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.496923 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.497055 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:12.500078 (+ 3023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.501325 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.505543 (+ 4218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.506258 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.506359 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.524596 (+ 18237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:12.524601 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:12.524604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:12.524646 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:12.528501 (+ 3855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.529204 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.529210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.529213 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.529379 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.530138 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.530143 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.530147 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.530476 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:12.530634 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.534075 (+ 3441us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.534299 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16927,"thread_start_us":16817,"threads_started":1}
I20260808 15:03:12.534657 13664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.496046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:50455 (local address 127.2.180.252:44587)
0808 15:03:12.510006 (+ 13960us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.510013 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.510036 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:03:12.510117 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.510122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.510217 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.510324 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:12.512805 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.514154 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.517335 (+ 3181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.518029 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.518157 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.524982 (+ 6825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:12.524990 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:12.524995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:12.525042 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:12.528500 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.531573 (+ 3073us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.531579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.531582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.531733 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.532220 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.532224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.532228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.532585 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:12.532724 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.534121 (+ 1397us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.534327 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13820,"thread_start_us":14880,"threads_started":1}
I20260808 15:03:12.535470 13653 heartbeater.cc:347] Connected to a master server at 127.2.180.252:44587
I20260808 15:03:12.535584 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.480260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.195:43939)
0808 15:03:12.491632 (+ 11372us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:12.491688 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:12.494226 (+ 2538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:12.499138 (+ 4912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:12.499143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:12.499609 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:12.499909 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.499921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.501535 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.501541 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.503731 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.503748 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.506762 (+ 3014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.506770 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.507093 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.508195 (+ 1102us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:12.508221 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:12.524398 (+ 16177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:12.528618 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:12.528623 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:12.528627 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:12.529085 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:12.529611 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:12.529615 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:12.529619 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:12.529791 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:12.533580 (+ 3789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:12.533585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:12.533776 (+ 191us) client_negotiation.cc:769] Sending connection context
0808 15:03:12.534865 (+ 1089us) client_negotiation.cc:240] Negotiation successful
0808 15:03:12.535164 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11044,"mutex_wait_us":3,"thread_start_us":14200,"threads_started":1}
I20260808 15:03:12.535637 13653 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:12.536265 13653 heartbeater.cc:511] Master 127.2.180.252:44587 requested a full tablet report, sending...
I20260808 15:03:12.536580 13654 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40807
I20260808 15:03:12.536711 13654 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:12.537096 13654 heartbeater.cc:511] Master 127.2.180.254:40807 requested a full tablet report, sending...
I20260808 15:03:12.537511 12962 ts_manager.cc:194] Registered new tserver with Master: 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331)
I20260808 15:03:12.537994 11759 ts_manager.cc:194] Re-registered known tserver with Master: 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331)
I20260808 15:03:12.539122 11759 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:43939
I20260808 15:03:12.550009 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.527462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58180 (local address 127.2.180.252:44587)
0808 15:03:12.530921 (+ 3459us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.530926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.531946 (+ 1020us) server_negotiation.cc:407] Connection header received
0808 15:03:12.531998 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.532002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.532068 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.532168 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:12.534735 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.537046 (+ 2311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.539092 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.539727 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.539864 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.544411 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:12.544416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:12.544420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:12.544449 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:12.547258 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.548042 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.548048 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.548051 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.548211 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.548612 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.548617 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.548620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.548947 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:12.549210 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.549546 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.549747 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3344,"thread_start_us":3222,"threads_started":1}
I20260808 15:03:12.551646 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.516011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35834 (local address 127.2.180.253:37237)
0808 15:03:12.523575 (+ 7564us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.523581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.531435 (+ 7854us) server_negotiation.cc:407] Connection header received
0808 15:03:12.531733 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.531738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.531852 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.531922 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:12.538295 (+ 6373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.539323 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.541579 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.542212 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.542334 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.545651 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:12.545656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:12.545659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:12.545687 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:12.548421 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.549176 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.549182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.549185 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.549325 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.549896 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.549900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.549904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.550262 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:12.550487 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.551057 (+ 570us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.551291 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6709,"thread_start_us":12583,"threads_started":1}
I20260808 15:03:12.552290 13673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.509873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58448 (local address 127.2.180.254:40807)
0808 15:03:12.520337 (+ 10464us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.520342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.526806 (+ 6464us) server_negotiation.cc:407] Connection header received
0808 15:03:12.529226 (+ 2420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.529231 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.529305 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.529404 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:12.538186 (+ 8782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.539357 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.541379 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.542057 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.542177 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.545640 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:12.545646 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:12.545651 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:12.545688 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:12.548807 (+ 3119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.549837 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.549842 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.549845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.549995 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:12.550620 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:12.550626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:12.550629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:12.550953 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:12.551192 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.551783 (+ 591us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.551990 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10339,"thread_start_us":10218,"threads_started":1}
I20260808 15:03:12.555717 11759 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58448:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:03:12.562410 11759 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:03:12.609400 13698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.581560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60966 (local address 127.2.180.193:44641)
0808 15:03:12.600383 (+ 18823us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.600392 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.601946 (+ 1554us) server_negotiation.cc:407] Connection header received
0808 15:03:12.602135 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.602143 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.602359 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.602794 (+ 435us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:12.603601 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.604835 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.606859 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.608208 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.608325 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.608536 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.608864 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.609029 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18641,"thread_start_us":18519,"threads_started":1}
I20260808 15:03:12.609520 13694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.580659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:44641 (local address 127.0.0.1:60966)
0808 15:03:12.601686 (+ 21027us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:12.601701 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:12.601966 (+ 265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:12.602774 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:12.602779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:12.602806 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:12.603245 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.603258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.604991 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.604996 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.606714 (+ 1718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.606726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.608406 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.608413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.608637 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.608690 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 15:03:12.608793 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 15:03:12.608963 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20915,"mutex_wait_us":62,"thread_start_us":17213,"threads_started":1}
I20260808 15:03:12.609549 13696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.580815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.2.180.195:33331)
0808 15:03:12.598853 (+ 18038us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.598862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.598895 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:03:12.598994 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.599002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.599277 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.599478 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:12.601396 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.603273 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.606894 (+ 3621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.608236 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.608359 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.608697 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.609083 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.609235 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17856,"thread_start_us":18982,"threads_started":1}
I20260808 15:03:12.610601 13695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.580221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:33331 (local address 127.0.0.1:41610)
0808 15:03:12.598156 (+ 17935us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:12.598174 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:12.598278 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:12.600477 (+ 2199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:12.600483 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:12.600514 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:12.601098 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.601113 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.603409 (+ 2296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.603415 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.605379 (+ 1964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.605397 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.608452 (+ 3055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.608458 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.608731 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.608895 (+ 164us) client_negotiation.cc:769] Sending connection context
0808 15:03:12.608977 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 15:03:12.609109 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17799,"thread_start_us":21211,"threads_started":1}
I20260808 15:03:12.611310 13692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.579790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41537 (local address 127.0.0.1:48344)
0808 15:03:12.596808 (+ 17018us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:12.597601 (+ 793us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:12.597927 (+ 326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:12.603859 (+ 5932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:12.603865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:12.603900 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:12.604301 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.604314 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.605739 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.605744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.607476 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.607491 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.610650 (+ 3159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.610656 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.610839 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.610870 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:03:12.610952 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 15:03:12.611052 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16884,"mutex_wait_us":5,"thread_start_us":16727,"threads_started":1}
I20260808 15:03:12.612169 13693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.580048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48344 (local address 127.2.180.194:41537)
0808 15:03:12.602436 (+ 22388us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.602444 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.602469 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:03:12.602561 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.602569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.603495 (+ 926us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.603847 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:12.604491 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.605603 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.607685 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.610041 (+ 2356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.610153 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.610293 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.611656 (+ 1363us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.611838 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22232,"thread_start_us":22134,"threads_started":1}
I20260808 15:03:12.614444 13556 tablet_service.cc:1467] Processing CreateTablet for tablet 9a5716096d854f2ab7238d2273519321 (DEFAULT_TABLE table=test-table [id=3f4ca92124f440b6bb49b7d70c98171f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:12.614471 13144 tablet_service.cc:1467] Processing CreateTablet for tablet 9a5716096d854f2ab7238d2273519321 (DEFAULT_TABLE table=test-table [id=3f4ca92124f440b6bb49b7d70c98171f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:12.615963 13556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a5716096d854f2ab7238d2273519321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:12.615993 13144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a5716096d854f2ab7238d2273519321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:12.615784 13355 tablet_service.cc:1467] Processing CreateTablet for tablet 9a5716096d854f2ab7238d2273519321 (DEFAULT_TABLE table=test-table [id=3f4ca92124f440b6bb49b7d70c98171f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:12.617070 13355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a5716096d854f2ab7238d2273519321. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:12.628575 13699 tablet_bootstrap.cc:492] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Bootstrap starting.
I20260808 15:03:12.631853 13700 tablet_bootstrap.cc:492] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Bootstrap starting.
I20260808 15:03:12.632612 13699 tablet_bootstrap.cc:654] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:12.632635 13703 tablet_bootstrap.cc:492] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Bootstrap starting.
I20260808 15:03:12.633869 13699 log.cc:824] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:12.635962 13703 tablet_bootstrap.cc:654] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:12.636891 13699 tablet_bootstrap.cc:492] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: No bootstrap required, opened a new log
I20260808 15:03:12.637177 13699 ts_tablet_manager.cc:1397] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 15:03:12.637200 13703 log.cc:824] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:12.638731 13700 tablet_bootstrap.cc:654] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:12.639931 13700 log.cc:824] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:12.641260 13703 tablet_bootstrap.cc:492] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: No bootstrap required, opened a new log
I20260808 15:03:12.641573 13703 ts_tablet_manager.cc:1397] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260808 15:03:12.642369 13700 tablet_bootstrap.cc:492] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: No bootstrap required, opened a new log
I20260808 15:03:12.642664 13700 ts_tablet_manager.cc:1397] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260808 15:03:12.647745 13699 raft_consensus.cc:348] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.648185 13699 raft_consensus.cc:374] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:12.648288 13699 raft_consensus.cc:729] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b2d76632a5c4aff888f5c3462ef9172, State: Initialized, Role: FOLLOWER
I20260808 15:03:12.648869 13699 consensus_queue.cc:260] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.649616 13700 raft_consensus.cc:348] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.650024 13654 heartbeater.cc:503] Master 127.2.180.254:40807 was elected leader, sending a full tablet report...
I20260808 15:03:12.650053 13700 raft_consensus.cc:374] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:12.650231 13700 raft_consensus.cc:729] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe501c99f632443e9ed883a3a33178e4, State: Initialized, Role: FOLLOWER
I20260808 15:03:12.650563 13699 ts_tablet_manager.cc:1428] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260808 15:03:12.650910 13700 consensus_queue.cc:260] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.652076 13703 raft_consensus.cc:348] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.652462 13703 raft_consensus.cc:374] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:12.652662 13703 raft_consensus.cc:729] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a62b1e0d7f4ba78262b35859df9bb4, State: Initialized, Role: FOLLOWER
I20260808 15:03:12.653194 13703 consensus_queue.cc:260] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.654970 13703 ts_tablet_manager.cc:1428] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260808 15:03:12.661160 13452 heartbeater.cc:503] Master 127.2.180.254:40807 was elected leader, sending a full tablet report...
I20260808 15:03:12.661898 13700 ts_tablet_manager.cc:1428] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Time spent starting tablet: real 0.019s user 0.008s sys 0.006s
W20260808 15:03:12.729161 13656 tablet.cc:2369] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:03:12.829567 13245 tablet.cc:2369] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:03:12.872797 13710 raft_consensus.cc:482] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:12.873082 13710 raft_consensus.cc:504] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.874672 13710 leader_election.cc:288] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331), a0a62b1e0d7f4ba78262b35859df9bb4 (127.2.180.193:44641)
I20260808 15:03:12.881776 13696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.874965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:41513 (local address 127.2.180.195:33331)
0808 15:03:12.875190 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.875197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.875955 (+ 758us) server_negotiation.cc:407] Connection header received
0808 15:03:12.876157 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.876162 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.876239 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.876332 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:12.876969 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.878103 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.880131 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.880917 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.881037 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.881091 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.881456 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.881544 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:03:12.881938 13717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.874852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:33331 (local address 127.2.180.194:41513)
0808 15:03:12.875836 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:12.875857 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:12.876021 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:12.876363 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:12.876366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:12.876387 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:12.876821 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.876838 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.878253 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.878258 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.879987 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.879999 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.881116 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.881123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.881304 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.881340 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:03:12.881426 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 15:03:12.881558 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":818,"mutex_wait_us":20,"thread_start_us":955,"threads_started":1}
I20260808 15:03:12.882988 13588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5716096d854f2ab7238d2273519321" candidate_uuid: "fe501c99f632443e9ed883a3a33178e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b2d76632a5c4aff888f5c3462ef9172" is_pre_election: true
I20260808 15:03:12.883476 13698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.875589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:60345 (local address 127.2.180.193:44641)
0808 15:03:12.875791 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.875817 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.876155 (+ 338us) server_negotiation.cc:407] Connection header received
0808 15:03:12.876304 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.876310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.876382 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.876464 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:12.877045 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.878100 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.881729 (+ 3629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.882674 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.882790 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.882838 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.883182 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.883265 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:03:12.883639 13715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.875079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:44641 (local address 127.2.180.194:60345)
0808 15:03:12.876065 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:12.876080 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:12.876191 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:12.876493 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:12.876496 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:12.876521 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:12.876892 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.876902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.879908 (+ 3006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.879913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.881571 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:12.881584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.882871 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.882877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:12.883039 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.883072 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:03:12.883169 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 15:03:12.883291 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":876,"mutex_wait_us":1,"thread_start_us":370,"threads_started":1}
I20260808 15:03:12.883635 13588 raft_consensus.cc:2454] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe501c99f632443e9ed883a3a33178e4 in term 0.
I20260808 15:03:12.884305 13288 leader_election.cc:302] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b2d76632a5c4aff888f5c3462ef9172, fe501c99f632443e9ed883a3a33178e4; no voters:
I20260808 15:03:12.884693 13710 raft_consensus.cc:2781] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:03:12.884589 13167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5716096d854f2ab7238d2273519321" candidate_uuid: "fe501c99f632443e9ed883a3a33178e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" is_pre_election: true
I20260808 15:03:12.884847 13710 raft_consensus.cc:482] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:03:12.884965 13710 raft_consensus.cc:3037] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:12.885089 13167 raft_consensus.cc:2454] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe501c99f632443e9ed883a3a33178e4 in term 0.
I20260808 15:03:12.887360 13710 raft_consensus.cc:504] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.887950 13710 leader_election.cc:288] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [CANDIDATE]: Term 1 election: Requested vote from peers 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331), a0a62b1e0d7f4ba78262b35859df9bb4 (127.2.180.193:44641)
I20260808 15:03:12.888327 13588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5716096d854f2ab7238d2273519321" candidate_uuid: "fe501c99f632443e9ed883a3a33178e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b2d76632a5c4aff888f5c3462ef9172"
I20260808 15:03:12.888399 13167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5716096d854f2ab7238d2273519321" candidate_uuid: "fe501c99f632443e9ed883a3a33178e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4"
I20260808 15:03:12.888535 13588 raft_consensus.cc:3037] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:12.888553 13167 raft_consensus.cc:3037] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:12.891604 13167 raft_consensus.cc:2454] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe501c99f632443e9ed883a3a33178e4 in term 1.
I20260808 15:03:12.891604 13588 raft_consensus.cc:2454] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe501c99f632443e9ed883a3a33178e4 in term 1.
I20260808 15:03:12.891986 13288 leader_election.cc:302] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b2d76632a5c4aff888f5c3462ef9172, fe501c99f632443e9ed883a3a33178e4; no voters:
I20260808 15:03:12.892268 13710 raft_consensus.cc:2781] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:12.892792 13710 raft_consensus.cc:686] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 1 LEADER]: Becoming Leader. State: Replica: fe501c99f632443e9ed883a3a33178e4, State: Running, Role: LEADER
I20260808 15:03:12.893560 13710 consensus_queue.cc:237] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:12.898274 11758 catalog_manager.cc:5495] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe501c99f632443e9ed883a3a33178e4 (127.2.180.194). New cstate: current_term: 1 leader_uuid: "fe501c99f632443e9ed883a3a33178e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } health_report { overall_health: UNKNOWN } } }
W20260808 15:03:12.905097 13456 tablet.cc:2369] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:03:12.953235 13693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:12.948729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.2.180.194:41537)
0808 15:03:12.948888 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:12.948892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:12.948908 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:03:12.948984 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:12.948987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:12.949059 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:12.949178 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:12.949462 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.950145 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.951366 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:12.951768 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:12.951915 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:12.952321 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:12.952815 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:12.952917 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:12.952975 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 15:03:12.953038 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:03:12.960978 13588 raft_consensus.cc:1261] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 1 FOLLOWER]: Refusing update from remote peer fe501c99f632443e9ed883a3a33178e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:12.961100 13167 raft_consensus.cc:1261] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 FOLLOWER]: Refusing update from remote peer fe501c99f632443e9ed883a3a33178e4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:12.962040 13721 consensus_queue.cc:1035] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:12.962406 13710 consensus_queue.cc:1035] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:12.972546 13725 mvcc.cc:205] Tried to move back new op lower bound from 7316280905558110208 to 7316280905295712256. Current Snapshot: MvccSnapshot[applied={T|T < 7316280905558110208}]
I20260808 15:03:12.983264 13727 mvcc.cc:205] Tried to move back new op lower bound from 7316280905558110208 to 7316280905295712256. Current Snapshot: MvccSnapshot[applied={T|T < 7316280905558110208}]
I20260808 15:03:12.997390 11758 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:12.997670 11758 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58448:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:13.005620 11758 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0bd40e8c8ed245cd862e7f502de9ae74: Sending DeleteTablet for 3 replicas of tablet 9a5716096d854f2ab7238d2273519321
I20260808 15:03:13.006634 13144 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a5716096d854f2ab7238d2273519321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:12 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:60966
I20260808 15:03:13.007102 13556 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a5716096d854f2ab7238d2273519321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:12 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:41610
I20260808 15:03:13.007443 13355 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a5716096d854f2ab7238d2273519321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:12 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:48344
I20260808 15:03:13.007568 13741 tablet_replica.cc:330] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: stopping tablet replica
I20260808 15:03:13.008037 13741 raft_consensus.cc:2229] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:13.008008 13143 tablet_service.cc:1514] Processing DeleteTablet for tablet 9a5716096d854f2ab7238d2273519321 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:12 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:60966
I20260808 15:03:13.008528 13741 raft_consensus.cc:2258] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:13.009506 13743 tablet_replica.cc:330] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: stopping tablet replica
I20260808 15:03:13.009876 13743 raft_consensus.cc:2229] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:03:13.010428 13743 raft_consensus.cc:2258] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:13.010703 13741 ts_tablet_manager.cc:1905] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:13.010991 13742 tablet_replica.cc:330] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: stopping tablet replica
I20260808 15:03:13.011389 13742 raft_consensus.cc:2229] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:13.012158 13742 raft_consensus.cc:2258] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:13.012804 13743 ts_tablet_manager.cc:1905] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:13.013947 13742 ts_tablet_manager.cc:1905] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:13.019572 13743 ts_tablet_manager.cc:1918] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:13.019605 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 11709
I20260808 15:03:13.019838 13743 log.cc:1196] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/9a5716096d854f2ab7238d2273519321
I20260808 15:03:13.020552 13742 ts_tablet_manager.cc:1918] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:13.020721 13742 log.cc:1196] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/9a5716096d854f2ab7238d2273519321
I20260808 15:03:13.020735 11806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:13.021245 13742 ts_tablet_manager.cc:1939] T 9a5716096d854f2ab7238d2273519321 P 3b2d76632a5c4aff888f5c3462ef9172: Deleting consensus metadata
I20260808 15:03:13.024000 13743 ts_tablet_manager.cc:1939] T 9a5716096d854f2ab7238d2273519321 P fe501c99f632443e9ed883a3a33178e4: Deleting consensus metadata
I20260808 15:03:13.035526 13741 ts_tablet_manager.cc:1918] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:13.035738 13741 log.cc:1196] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/9a5716096d854f2ab7238d2273519321
I20260808 15:03:13.036310 13741 ts_tablet_manager.cc:1939] T 9a5716096d854f2ab7238d2273519321 P a0a62b1e0d7f4ba78262b35859df9bb4: Deleting consensus metadata
I20260808 15:03:13.043298 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:13.010015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.2.180.253:37237)
0808 15:03:13.010188 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:13.010193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:13.019259 (+ 9066us) server_negotiation.cc:407] Connection header received
0808 15:03:13.021693 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:13.021698 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:13.021797 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:13.021900 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:13.025841 (+ 3941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:13.027020 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:13.029025 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:13.029694 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:13.029818 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:13.036347 (+ 6529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:13.036354 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:13.036359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:13.036400 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:13.039976 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:13.040763 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:13.040769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:13.040773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:13.040937 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:13.041349 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:13.041355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:13.041358 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:13.041727 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:13.041997 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:13.042594 (+ 597us) server_negotiation.cc:299] Negotiation successful
0808 15:03:13.042970 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:03:13.059635 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:13.018339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58192 (local address 127.2.180.252:44587)
0808 15:03:13.018542 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:13.018549 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:13.043247 (+ 24698us) server_negotiation.cc:407] Connection header received
0808 15:03:13.043488 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:13.043494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:13.043577 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:13.043650 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:13.046228 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:13.047333 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:13.049170 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:13.049811 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:13.049925 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:13.053183 (+ 3258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:13.053189 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:13.053194 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:13.053236 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:13.056669 (+ 3433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:13.057556 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:13.057562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:13.057566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:13.057750 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:13.058202 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:13.058207 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:13.058211 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:13.058548 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:13.058775 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:13.059096 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 15:03:13.059306 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:03:13.146826 11737 catalog_manager.cc:4841] TS fe501c99f632443e9ed883a3a33178e4 (127.2.180.194:41537): tablet 9a5716096d854f2ab7238d2273519321 (table test-table [id=3f4ca92124f440b6bb49b7d70c98171f]) successfully deleted
I20260808 15:03:13.147270 11734 catalog_manager.cc:4841] TS 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331): tablet 9a5716096d854f2ab7238d2273519321 (table test-table [id=3f4ca92124f440b6bb49b7d70c98171f]) successfully deleted
I20260808 15:03:13.148052 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 11709
W20260808 15:03:13.182055 13077 connection.cc:535] server connection from 127.0.0.1:60966 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 15:03:13.217301 13658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:13.187670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35858 (local address 127.2.180.253:37237)
0808 15:03:13.188025 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:13.188032 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:13.200771 (+ 12739us) server_negotiation.cc:407] Connection header received
0808 15:03:13.201081 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:13.201087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:13.201178 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:13.201252 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:13.203627 (+ 2375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:13.204846 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:13.206785 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:13.207527 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:13.207647 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:13.210568 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:13.210575 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:13.210580 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:13.210620 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:13.214157 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:13.214908 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:13.214914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:13.214917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:13.215104 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:13.215488 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:13.215493 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:13.215496 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:13.215893 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:13.216590 (+ 697us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:13.216728 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 15:03:13.216952 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:03:13.218334 13681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:13.200521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58196 (local address 127.2.180.252:44587)
0808 15:03:13.200700 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:13.200707 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:13.202684 (+ 1977us) server_negotiation.cc:407] Connection header received
0808 15:03:13.202911 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:13.202916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:13.202994 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:13.203077 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:13.205137 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:13.205876 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:13.207620 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:13.208295 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:13.208420 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:13.212621 (+ 4201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:13.212627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:13.212630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:13.212666 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:13.215502 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:13.216175 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:13.216181 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:13.216184 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:13.216335 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:13.216805 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:13.216811 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:13.216813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:13.217151 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:13.217327 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:13.217864 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 15:03:13.218045 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:03:13.979713 13759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:13.979023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.195:39993)
0808 15:03:13.979427 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:13.979539 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":202,"threads_started":1}
W20260808 15:03:13.980262 13654 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:40807 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
I20260808 15:03:13.987455 13760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:13.986813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.194:51439)
0808 15:03:13.987202 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:13.987312 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":188,"threads_started":1}
W20260808 15:03:13.987833 13452 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:40807 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
I20260808 15:03:14.007750 13761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.007129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.193:39213)
0808 15:03:14.007489 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.007601 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":168,"threads_started":1}
W20260808 15:03:14.008105 13243 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:40807 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
I20260808 15:03:14.505874 13763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bd40e8c8ed245cd862e7f502de9ae74)
I20260808 15:03:14.506181 13763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } }
I20260808 15:03:14.507522 13763 leader_election.cc:288] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd40e8c8ed245cd862e7f502de9ae74 (127.2.180.254:40807), 7b67415abc55416c87a38d8fe934b740 (127.2.180.253:37237)
I20260808 15:03:14.509327 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.507584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58858)
0808 15:03:14.508889 (+ 1305us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.509089 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1190,"thread_start_us":1024,"threads_started":1}
W20260808 15:03:14.509608 12932 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
I20260808 15:03:14.509822 13768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0bd40e8c8ed245cd862e7f502de9ae74)
I20260808 15:03:14.510093 13768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } }
I20260808 15:03:14.511235 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.510860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58868)
0808 15:03:14.511008 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.511088 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 15:03:14.511583 13768 leader_election.cc:288] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0bd40e8c8ed245cd862e7f502de9ae74 (127.2.180.254:40807), 216720daa1104240aa8aa7ba1c268acd (127.2.180.252:44587)
W20260808 15:03:14.511520 12932 leader_election.cc:334] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0bd40e8c8ed245cd862e7f502de9ae74 (127.2.180.254:40807): Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
I20260808 15:03:14.513020 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.511621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58884)
0808 15:03:14.512474 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.512750 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":238,"threads_started":1}
W20260808 15:03:14.513790 12644 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
I20260808 15:03:14.515499 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.515076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58898)
0808 15:03:14.515283 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.515364 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260808 15:03:14.515661 12644 leader_election.cc:334] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0bd40e8c8ed245cd862e7f502de9ae74 (127.2.180.254:40807): Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
I20260808 15:03:14.516615 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.508573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:37237 (local address 127.0.0.1:35872)
0808 15:03:14.509099 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.509113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:14.509264 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:14.509590 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:14.509595 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:14.509621 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:14.509941 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.509953 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.511400 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.511405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.514448 (+ 3043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.514461 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.515995 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.516002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.516135 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.516161 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 15:03:14.516255 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 15:03:14.516344 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":271,"threads_started":1}
I20260808 15:03:14.516791 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.508031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.2.180.253:37237)
0808 15:03:14.508581 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.508587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.509195 (+ 608us) server_negotiation.cc:407] Connection header received
0808 15:03:14.509393 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.509398 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.509470 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.509568 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:14.510120 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.511232 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.514695 (+ 3463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.515833 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.515965 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.516076 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.516293 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.516427 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":241,"threads_started":1}
I20260808 15:03:14.517398 12686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "216720daa1104240aa8aa7ba1c268acd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7b67415abc55416c87a38d8fe934b740" is_pre_election: true
I20260808 15:03:14.517715 12686 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 216720daa1104240aa8aa7ba1c268acd in term 1.
I20260808 15:03:14.518194 12937 leader_election.cc:302] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 216720daa1104240aa8aa7ba1c268acd, 7b67415abc55416c87a38d8fe934b740; no voters: 0bd40e8c8ed245cd862e7f502de9ae74
I20260808 15:03:14.518582 13763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 15:03:14.518733 13763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0bd40e8c8ed245cd862e7f502de9ae74)
I20260808 15:03:14.518836 13763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:03:14.520100 13773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.511759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44587 (local address 127.0.0.1:58208)
0808 15:03:14.513335 (+ 1576us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.513358 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:14.513481 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:14.513752 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:14.513757 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:14.513785 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:14.514227 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.514240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.515820 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.515826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.517988 (+ 2162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.518001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.519500 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.519505 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.519626 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.519650 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:03:14.519722 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 15:03:14.519833 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1467,"mutex_wait_us":1,"thread_start_us":820,"threads_started":1}
I20260808 15:03:14.520262 13777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.512479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58208 (local address 127.2.180.252:44587)
0808 15:03:14.513032 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.513036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.513435 (+ 399us) server_negotiation.cc:407] Connection header received
0808 15:03:14.513577 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.513580 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.513635 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.513743 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:14.514463 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.515627 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.518139 (+ 2512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.519343 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.519461 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.519568 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.519763 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.519913 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":306,"threads_started":1}
I20260808 15:03:14.520843 12984 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7b67415abc55416c87a38d8fe934b740" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "216720daa1104240aa8aa7ba1c268acd" is_pre_election: true
I20260808 15:03:14.522040 13763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } }
I20260808 15:03:14.522554 12984 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7b67415abc55416c87a38d8fe934b740 in current term 2: Already voted for candidate 216720daa1104240aa8aa7ba1c268acd in this term.
I20260808 15:03:14.522846 13763 leader_election.cc:288] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [CANDIDATE]: Term 2 election: Requested vote from peers 0bd40e8c8ed245cd862e7f502de9ae74 (127.2.180.254:40807), 7b67415abc55416c87a38d8fe934b740 (127.2.180.253:37237)
I20260808 15:03:14.523072 12686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "216720daa1104240aa8aa7ba1c268acd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7b67415abc55416c87a38d8fe934b740"
I20260808 15:03:14.523227 12686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:03:14.523334 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.522738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58912)
0808 15:03:14.523074 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.523161 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218}
I20260808 15:03:14.524386 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.523990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58928)
0808 15:03:14.524137 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.524229 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 15:03:14.524290 12648 leader_election.cc:302] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7b67415abc55416c87a38d8fe934b740; no voters: 0bd40e8c8ed245cd862e7f502de9ae74, 216720daa1104240aa8aa7ba1c268acd
W20260808 15:03:14.524547 12932 leader_election.cc:334] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 0bd40e8c8ed245cd862e7f502de9ae74 (127.2.180.254:40807): Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
I20260808 15:03:14.525750 12686 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 216720daa1104240aa8aa7ba1c268acd in term 2.
I20260808 15:03:14.525926 13768 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:03:14.526949 12937 leader_election.cc:302] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 216720daa1104240aa8aa7ba1c268acd, 7b67415abc55416c87a38d8fe934b740; no voters: 0bd40e8c8ed245cd862e7f502de9ae74
I20260808 15:03:14.527318 13763 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:03:14.527787 13763 raft_consensus.cc:686] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [term 2 LEADER]: Becoming Leader. State: Replica: 216720daa1104240aa8aa7ba1c268acd, State: Running, Role: LEADER
I20260808 15:03:14.528188 13763 consensus_queue.cc:237] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } }
I20260808 15:03:14.529927 13779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 216720daa1104240aa8aa7ba1c268acd. Latest consensus state: current_term: 2 leader_uuid: "216720daa1104240aa8aa7ba1c268acd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:14.530339 13779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:14.530946 13782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:14.534437 13782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:14.535223 13782 catalog_manager.cc:1259] Loaded cluster ID: c4bb6cc0d5124193a8a0698d89d499cd
I20260808 15:03:14.535293 13782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:14.536437 13782 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:14.537206 13782 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Loaded TSK: 0
I20260808 15:03:14.537978 13782 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:03:14.583506 12962 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:03:14.590368 12962 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:03:14.594216 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.593838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58946)
0808 15:03:14.593994 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.594086 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 15:03:14.594254 12686 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [term 2 FOLLOWER]: Refusing update from remote peer 216720daa1104240aa8aa7ba1c268acd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260808 15:03:14.595077 13779 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [LEADER]: Connected to new peer: Peer: permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 15:03:14.595288 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.594825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58960)
0808 15:03:14.594995 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.595087 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 15:03:14.595446 12932 consensus_peers.cc:487] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd -> Peer 0bd40e8c8ed245cd862e7f502de9ae74 (127.2.180.254:40807): Couldn't send request to peer 0bd40e8c8ed245cd862e7f502de9ae74. Status: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 15:03:14.597683 13768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 216720daa1104240aa8aa7ba1c268acd. Latest consensus state: current_term: 2 leader_uuid: "216720daa1104240aa8aa7ba1c268acd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:14.597925 13768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:03:14.599264 13779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "216720daa1104240aa8aa7ba1c268acd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:14.599562 13779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:14.600962 13768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "216720daa1104240aa8aa7ba1c268acd" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "0bd40e8c8ed245cd862e7f502de9ae74" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40807 } } peers { permanent_uuid: "7b67415abc55416c87a38d8fe934b740" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 37237 } attrs { promote: false } } peers { permanent_uuid: "216720daa1104240aa8aa7ba1c268acd" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44587 } attrs { promote: false } } }
I20260808 15:03:14.601259 13768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b67415abc55416c87a38d8fe934b740 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:03:14.601727 13784 mvcc.cc:205] Tried to move back new op lower bound from 7316280912252260352 to 7316280911990669312. Current Snapshot: MvccSnapshot[applied={T|T < 7316280912252260352 or (T in {7316280912252260352})}]
I20260808 15:03:14.615228 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.606980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41537 (local address 127.0.0.1:48366)
0808 15:03:14.607112 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.607130 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:14.607234 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:14.608489 (+ 1255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:14.608494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:14.608521 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:14.608723 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.608735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.610256 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.610261 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.611793 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.611827 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.614476 (+ 2649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.614482 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.614746 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.614802 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 15:03:14.614886 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 15:03:14.615010 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":4}
I20260808 15:03:14.616202 13765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.607674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:44641 (local address 127.0.0.1:60980)
0808 15:03:14.607838 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.607858 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:14.607961 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:14.608718 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:14.608723 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:14.608746 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:14.609044 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.609056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.610475 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.610480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.612930 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.612945 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.614571 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.614577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.614761 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.615730 (+ 969us) client_negotiation.cc:769] Sending connection context
0808 15:03:14.615795 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 15:03:14.615945 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":1768}
I20260808 15:03:14.616330 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.607978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.2.180.193:44641)
0808 15:03:14.608401 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.608406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.608423 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:03:14.608489 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.608494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.608558 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.609182 (+ 624us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:14.609342 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.610336 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.613075 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.614342 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.614836 (+ 494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.614939 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.615863 (+ 924us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.616000 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":446,"threads_started":1}
I20260808 15:03:14.617159 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.607660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48366 (local address 127.2.180.194:41537)
0808 15:03:14.608140 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.608155 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.608177 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:03:14.608252 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.608258 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.608331 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.608872 (+ 541us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:14.609042 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.610091 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.613147 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.614300 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.616582 (+ 2282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.616703 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.616764 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.616898 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":256,"threads_started":1}
I20260808 15:03:14.617321 13144 tablet_service.cc:1467] Processing CreateTablet for tablet 518bb805ef9143098c8fc696eb13a463 (DEFAULT_TABLE table=test-table [id=67d9605a6e10470cafe0d42a06d80516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:14.617764 13144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518bb805ef9143098c8fc696eb13a463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:14.617795 13355 tablet_service.cc:1467] Processing CreateTablet for tablet 518bb805ef9143098c8fc696eb13a463 (DEFAULT_TABLE table=test-table [id=67d9605a6e10470cafe0d42a06d80516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:14.618189 13355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518bb805ef9143098c8fc696eb13a463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:14.628051 13797 tablet_bootstrap.cc:492] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: Bootstrap starting.
I20260808 15:03:14.629882 13797 tablet_bootstrap.cc:654] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:14.632176 13798 tablet_bootstrap.cc:492] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: Bootstrap starting.
I20260808 15:03:14.633692 13797 tablet_bootstrap.cc:492] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: No bootstrap required, opened a new log
I20260808 15:03:14.633831 13797 ts_tablet_manager.cc:1397] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260808 15:03:14.634397 13798 tablet_bootstrap.cc:654] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:14.634614 13797 raft_consensus.cc:348] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.634846 13797 raft_consensus.cc:374] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:14.634943 13797 raft_consensus.cc:729] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe501c99f632443e9ed883a3a33178e4, State: Initialized, Role: FOLLOWER
I20260808 15:03:14.635165 13797 consensus_queue.cc:260] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.636051 13797 ts_tablet_manager.cc:1428] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260808 15:03:14.636493 13798 tablet_bootstrap.cc:492] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: No bootstrap required, opened a new log
I20260808 15:03:14.636629 13798 ts_tablet_manager.cc:1397] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 15:03:14.637353 13798 raft_consensus.cc:348] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.637596 13798 raft_consensus.cc:374] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:14.637686 13798 raft_consensus.cc:729] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a62b1e0d7f4ba78262b35859df9bb4, State: Initialized, Role: FOLLOWER
I20260808 15:03:14.637902 13798 consensus_queue.cc:260] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.638134 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.608303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:33331 (local address 127.0.0.1:41620)
0808 15:03:14.615434 (+ 7131us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.615451 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:14.615536 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:14.632332 (+ 16796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:14.632337 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:14.632363 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:14.632544 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.632556 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.633729 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.633734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.635096 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.635112 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.637357 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.637363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.637573 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.637606 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:03:14.637689 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 15:03:14.637811 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7038,"thread_start_us":16533,"threads_started":1}
I20260808 15:03:14.638800 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.636665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.194:47487)
0808 15:03:14.637349 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.637485 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":449,"threads_started":1}
I20260808 15:03:14.639108 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.608456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.2.180.195:33331)
0808 15:03:14.631982 (+ 23526us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.631989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.632008 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:03:14.632090 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.632094 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.632178 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.632315 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:14.632685 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.633568 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.635280 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.636314 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.638319 (+ 2005us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.638426 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.638475 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.638602 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23401,"thread_start_us":23258,"threads_started":1}
I20260808 15:03:14.639832 13556 tablet_service.cc:1467] Processing CreateTablet for tablet 518bb805ef9143098c8fc696eb13a463 (DEFAULT_TABLE table=test-table [id=67d9605a6e10470cafe0d42a06d80516]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:14.640282 13556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 518bb805ef9143098c8fc696eb13a463. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:14.643791 13450 heartbeater.cc:503] Master 127.2.180.252:44587 was elected leader, sending a full tablet report...
I20260808 15:03:14.644555 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.644084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.194:53233)
0808 15:03:14.644317 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.644409 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260808 15:03:14.647051 13806 tablet_bootstrap.cc:492] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: Bootstrap starting.
I20260808 15:03:14.647724 13798 ts_tablet_manager.cc:1428] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260808 15:03:14.648828 13806 tablet_bootstrap.cc:654] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:14.649138 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.647853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.193:59447)
0808 15:03:14.648705 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.648857 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":728,"thread_start_us":608,"threads_started":1}
I20260808 15:03:14.649358 13242 heartbeater.cc:503] Master 127.2.180.252:44587 was elected leader, sending a full tablet report...
I20260808 15:03:14.652665 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.649799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.193:46781)
0808 15:03:14.652247 (+ 2448us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.652446 (+ 199us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2328,"thread_start_us":2188,"threads_started":1}
I20260808 15:03:14.653440 13806 tablet_bootstrap.cc:492] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: No bootstrap required, opened a new log
I20260808 15:03:14.653554 13806 ts_tablet_manager.cc:1397] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 15:03:14.654105 13806 raft_consensus.cc:348] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.654299 13806 raft_consensus.cc:374] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:14.654366 13806 raft_consensus.cc:729] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b2d76632a5c4aff888f5c3462ef9172, State: Initialized, Role: FOLLOWER
I20260808 15:03:14.654531 13806 consensus_queue.cc:260] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.655154 13806 ts_tablet_manager.cc:1428] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 15:03:14.656433 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.655454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.195:54619)
0808 15:03:14.656052 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.656194 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":303,"threads_started":1}
I20260808 15:03:14.657212 13653 heartbeater.cc:503] Master 127.2.180.252:44587 was elected leader, sending a full tablet report...
I20260808 15:03:14.657660 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.657339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.195:60327)
0808 15:03:14.657460 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.657529 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260808 15:03:14.835599 13805 raft_consensus.cc:482] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:14.835842 13805 raft_consensus.cc:504] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.837437 13805 leader_election.cc:288] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe501c99f632443e9ed883a3a33178e4 (127.2.180.194:41537), 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331)
I20260808 15:03:14.843102 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.837790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:39897 (local address 127.2.180.195:33331)
0808 15:03:14.837996 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.838003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.838076 (+ 73us) server_negotiation.cc:407] Connection header received
0808 15:03:14.838281 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.838285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.838345 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.838418 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:14.838927 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.839637 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.841451 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.842223 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.842383 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.842457 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.842733 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.842862 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:03:14.843258 13807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.837664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:33331 (local address 127.2.180.193:39897)
0808 15:03:14.837967 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.837987 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:14.838139 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:14.838447 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:14.838452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:14.838475 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:14.838797 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.838806 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.839766 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.839770 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.841299 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.841312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.842396 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.842402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.842563 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.842591 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 15:03:14.842692 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 15:03:14.842851 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":6,"spinlock_wait_cycles":5376}
I20260808 15:03:14.843328 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.837888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:56325 (local address 127.2.180.194:41537)
0808 15:03:14.838044 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.838050 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.838070 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:03:14.838295 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.838300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.838357 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.838431 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:14.838941 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.839618 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.841524 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.842394 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.842567 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.842643 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.842941 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.843062 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:03:14.843387 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.837664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41537 (local address 127.2.180.193:56325)
0808 15:03:14.837982 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.838000 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:14.838144 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:14.838461 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:14.838465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:14.838487 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:14.838800 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.838809 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.839745 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.839751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.841384 (+ 1633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:14.841400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.842608 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.842618 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:14.842802 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.842825 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:03:14.842903 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 15:03:14.843098 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":116,"spinlock_wait_cycles":768}
I20260808 15:03:14.844069 13383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518bb805ef9143098c8fc696eb13a463" candidate_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe501c99f632443e9ed883a3a33178e4" is_pre_election: true
I20260808 15:03:14.844169 13588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518bb805ef9143098c8fc696eb13a463" candidate_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b2d76632a5c4aff888f5c3462ef9172" is_pre_election: true
I20260808 15:03:14.844390 13588 raft_consensus.cc:2454] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0a62b1e0d7f4ba78262b35859df9bb4 in term 0.
I20260808 15:03:14.844427 13383 raft_consensus.cc:2454] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0a62b1e0d7f4ba78262b35859df9bb4 in term 0.
I20260808 15:03:14.844859 13072 leader_election.cc:302] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b2d76632a5c4aff888f5c3462ef9172, a0a62b1e0d7f4ba78262b35859df9bb4; no voters:
I20260808 15:03:14.845196 13805 raft_consensus.cc:2781] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:03:14.845304 13805 raft_consensus.cc:482] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:03:14.845388 13805 raft_consensus.cc:3037] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:14.847453 13805 raft_consensus.cc:504] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.848031 13805 leader_election.cc:288] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [CANDIDATE]: Term 1 election: Requested vote from peers fe501c99f632443e9ed883a3a33178e4 (127.2.180.194:41537), 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331)
I20260808 15:03:14.848407 13383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518bb805ef9143098c8fc696eb13a463" candidate_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe501c99f632443e9ed883a3a33178e4"
I20260808 15:03:14.848596 13383 raft_consensus.cc:3037] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:14.848531 13588 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "518bb805ef9143098c8fc696eb13a463" candidate_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b2d76632a5c4aff888f5c3462ef9172"
I20260808 15:03:14.849944 13588 raft_consensus.cc:3037] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:14.851640 13383 raft_consensus.cc:2454] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0a62b1e0d7f4ba78262b35859df9bb4 in term 1.
I20260808 15:03:14.852047 13077 leader_election.cc:302] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0a62b1e0d7f4ba78262b35859df9bb4, fe501c99f632443e9ed883a3a33178e4; no voters:
I20260808 15:03:14.852313 13805 raft_consensus.cc:2781] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:14.852921 13588 raft_consensus.cc:2454] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0a62b1e0d7f4ba78262b35859df9bb4 in term 1.
I20260808 15:03:14.853544 13805 raft_consensus.cc:686] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 LEADER]: Becoming Leader. State: Replica: a0a62b1e0d7f4ba78262b35859df9bb4, State: Running, Role: LEADER
I20260808 15:03:14.853992 13805 consensus_queue.cc:237] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } }
I20260808 15:03:14.854218 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.853852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.193:49163)
0808 15:03:14.854007 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.854081 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 15:03:14.857875 12962 catalog_manager.cc:5495] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0a62b1e0d7f4ba78262b35859df9bb4 (127.2.180.193). New cstate: current_term: 1 leader_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0a62b1e0d7f4ba78262b35859df9bb4" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 44641 } health_report { overall_health: HEALTHY } } }
I20260808 15:03:14.921310 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.916355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.2.180.193:44641)
0808 15:03:14.916555 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.916559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.916582 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:03:14.916780 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.916783 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.916877 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.917009 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:14.917291 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.918056 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.919382 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.919783 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.919907 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.920255 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:14.920780 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:14.920881 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.920970 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.921036 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:03:14.924957 13588 raft_consensus.cc:1261] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 1 FOLLOWER]: Refusing update from remote peer a0a62b1e0d7f4ba78262b35859df9bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:14.925323 13383 raft_consensus.cc:1261] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 1 FOLLOWER]: Refusing update from remote peer a0a62b1e0d7f4ba78262b35859df9bb4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:14.925727 13813 consensus_queue.cc:1035] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b2d76632a5c4aff888f5c3462ef9172" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33331 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:14.926184 13805 consensus_queue.cc:1035] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe501c99f632443e9ed883a3a33178e4" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:14.928241 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.927794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.195:34919)
0808 15:03:14.927993 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.928108 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 15:03:14.928915 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.928540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.194:38417)
0808 15:03:14.928685 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.928791 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 15:03:14.933068 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.932578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.193:59069)
0808 15:03:14.932811 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.932905 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 15:03:14.933619 13820 mvcc.cc:205] Tried to move back new op lower bound from 7316280913606451200 to 7316280913326247936. Current Snapshot: MvccSnapshot[applied={T|T < 7316280913606451200}]
I20260808 15:03:14.934584 13811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.934154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.195:60355)
0808 15:03:14.934352 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.934433 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260808 15:03:14.944085 13809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.943539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.193:47283)
0808 15:03:14.943789 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.943902 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260808 15:03:14.944433 13803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.944001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.2.180.194:42533)
0808 15:03:14.944213 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:14.944297 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
I20260808 15:03:14.948866 12961 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:14.949118 12961 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:14.954100 12961 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 216720daa1104240aa8aa7ba1c268acd: Sending DeleteTablet for 3 replicas of tablet 518bb805ef9143098c8fc696eb13a463
I20260808 15:03:14.955003 13355 tablet_service.cc:1514] Processing DeleteTablet for tablet 518bb805ef9143098c8fc696eb13a463 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:14 UTC) from {username='oryx', principal='oryx/127.2.180.252@KRBTEST.COM'} at 127.0.0.1:48366
I20260808 15:03:14.955242 13144 tablet_service.cc:1514] Processing DeleteTablet for tablet 518bb805ef9143098c8fc696eb13a463 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:14 UTC) from {username='oryx', principal='oryx/127.2.180.252@KRBTEST.COM'} at 127.0.0.1:60980
I20260808 15:03:14.955933 13834 tablet_replica.cc:330] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: stopping tablet replica
I20260808 15:03:14.956085 13556 tablet_service.cc:1514] Processing DeleteTablet for tablet 518bb805ef9143098c8fc696eb13a463 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:14 UTC) from {username='oryx', principal='oryx/127.2.180.252@KRBTEST.COM'} at 127.0.0.1:41620
I20260808 15:03:14.956246 13834 raft_consensus.cc:2229] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:03:14.956420 13833 tablet_replica.cc:330] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: stopping tablet replica
I20260808 15:03:14.956652 13833 raft_consensus.cc:2229] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:14.956948 13833 raft_consensus.cc:2258] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:14.956933 13834 raft_consensus.cc:2258] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:14.957132 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 12938
I20260808 15:03:14.958212 13834 ts_tablet_manager.cc:1905] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:14.958302 13833 ts_tablet_manager.cc:1905] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:14.959833 13835 tablet_replica.cc:330] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: stopping tablet replica
I20260808 15:03:14.960011 13835 raft_consensus.cc:2229] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:14.960302 13835 raft_consensus.cc:2258] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:14.961351 13835 ts_tablet_manager.cc:1905] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:14.964506 13833 ts_tablet_manager.cc:1918] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:14.964641 13833 log.cc:1196] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/518bb805ef9143098c8fc696eb13a463
I20260808 15:03:14.965097 13833 ts_tablet_manager.cc:1939] T 518bb805ef9143098c8fc696eb13a463 P fe501c99f632443e9ed883a3a33178e4: Deleting consensus metadata
I20260808 15:03:14.965513 13834 ts_tablet_manager.cc:1918] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:14.965713 13834 log.cc:1196] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/518bb805ef9143098c8fc696eb13a463
I20260808 15:03:14.966215 13834 ts_tablet_manager.cc:1939] T 518bb805ef9143098c8fc696eb13a463 P a0a62b1e0d7f4ba78262b35859df9bb4: Deleting consensus metadata
I20260808 15:03:14.967342 12933 catalog_manager.cc:4841] TS a0a62b1e0d7f4ba78262b35859df9bb4 (127.2.180.193:44641): tablet 518bb805ef9143098c8fc696eb13a463 (table test-table [id=67d9605a6e10470cafe0d42a06d80516]) successfully deleted
I20260808 15:03:14.968010 13835 ts_tablet_manager.cc:1918] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:14.968222 13835 log.cc:1196] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/518bb805ef9143098c8fc696eb13a463
I20260808 15:03:14.968725 13835 ts_tablet_manager.cc:1939] T 518bb805ef9143098c8fc696eb13a463 P 3b2d76632a5c4aff888f5c3462ef9172: Deleting consensus metadata
I20260808 15:03:14.969772 12932 catalog_manager.cc:4841] TS 3b2d76632a5c4aff888f5c3462ef9172 (127.2.180.195:33331): tablet 518bb805ef9143098c8fc696eb13a463 (table test-table [id=67d9605a6e10470cafe0d42a06d80516]) successfully deleted
I20260808 15:03:14.969799 12937 catalog_manager.cc:4841] TS fe501c99f632443e9ed883a3a33178e4 (127.2.180.194:41537): tablet 518bb805ef9143098c8fc696eb13a463 (table test-table [id=67d9605a6e10470cafe0d42a06d80516]) successfully deleted
I20260808 15:03:14.983312 13794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:14.958077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32770 (local address 127.2.180.193:44641)
0808 15:03:14.958230 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:14.958245 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:14.968869 (+ 10624us) server_negotiation.cc:407] Connection header received
0808 15:03:14.969078 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:14.969084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:14.969178 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:14.969279 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:14.971276 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.972164 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.973957 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:14.974402 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:14.974494 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:14.977639 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:14.977650 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:14.977665 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:14.977702 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:14.980583 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:14.981412 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:14.981421 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:14.981424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:14.981586 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:14.981961 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:14.981965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:14.981966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:14.982329 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:14.982549 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:14.982865 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 15:03:14.983014 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260808 15:03:14.983938 13232 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:15.081506 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 12938
I20260808 15:03:15.107497 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 13256
I20260808 15:03:15.120590 13793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:15.108268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48380 (local address 127.2.180.194:41537)
0808 15:03:15.108451 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:15.108457 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:15.108541 (+ 84us) server_negotiation.cc:407] Connection header received
0808 15:03:15.108753 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:15.108759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:15.108826 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:15.108912 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:15.110202 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:15.110936 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:15.112257 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:15.112675 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:15.112782 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:15.115050 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:15.115066 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:15.115084 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:15.115131 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:15.118122 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:15.118724 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:15.118731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:15.118733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:15.118865 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:15.119184 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:15.119188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:15.119190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:15.119538 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:15.119787 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:15.120107 (+ 320us) server_negotiation.cc:299] Negotiation successful
0808 15:03:15.120284 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:03:15.121284 13441 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:15.185725 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:15.185051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58976)
0808 15:03:15.185467 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:15.185550 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":210,"threads_started":1}
I20260808 15:03:15.187037 13843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:15.186714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40807 (local address 127.0.0.1:58980)
0808 15:03:15.186855 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:15.186931 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:40807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 15:03:15.245278 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 13256
I20260808 15:03:15.270961 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 13469
I20260808 15:03:15.286159 13796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:15.271671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41622 (local address 127.2.180.195:33331)
0808 15:03:15.271878 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:15.271882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:15.271900 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:03:15.272028 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:15.272032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:15.272096 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:15.272185 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:15.273761 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:15.274838 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:15.276733 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:15.277340 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:15.277432 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:15.281030 (+ 3598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:15.281049 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:15.281070 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:15.281108 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:15.283784 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:15.284558 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:15.284566 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:15.284569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:15.284692 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:15.284978 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:15.284981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:15.284983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:15.285344 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:15.285556 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:15.285748 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 15:03:15.285901 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:03:15.286738 13645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:15.377017 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 13469
I20260808 15:03:15.400781 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 12469
I20260808 15:03:15.401825 12711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:15.517082 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 12469
I20260808 15:03:15.541179 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 12613
I20260808 15:03:15.542286 13014 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:15.661186 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 12613
2026-08-08T15:03:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (9583 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:03:15 dist-test-slave-jlzn krb5kdc[13853](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:03:15 dist-test-slave-jlzn krb5kdc[13853](info): set up 2 sockets
Aug 08 15:03:15 dist-test-slave-jlzn krb5kdc[13853](info): commencing operation
krb5kdc: starting...
W20260808 15:03:16.835703 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:03:16 dist-test-slave-jlzn krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:03:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.254.keytab.
Entry for principal oryx/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.254.keytab.
I20260808 15:03:17.035030 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40235
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.254.keytab
--principal=oryx/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:17.342998 13870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:17.343268 13870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:17.343362 13870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:17.350195 13870 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:17.350281 13870 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 15:03:17.350327 13870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:17.350386 13870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:17.350435 13870 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:17.350495 13870 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:17.359619 13870 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40235
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.254.keytab
--principal=oryx/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:17.360707 13870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:17.361865 13870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:17.372710 13878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:17.372751 13876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:17.372787 13881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:17.373356 13870 server_base.cc:1034] running on GCE node
I20260808 15:03:17.374075 13870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:17.375279 13870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:17.376492 13870 hybrid_clock.cc:648] HybridClock initialized: now 1786201397376428 us; error 34 us; skew 500 ppm
I20260808 15:03:17.376780 13870 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:17 dist-test-slave-jlzn krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201397, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:17.379774 13870 init.cc:377] Logged in from keytab as oryx/127.2.180.254@KRBTEST.COM (short username oryx)
I20260808 15:03:17.381063 13870 webserver.cc:460] Webserver started at http://127.2.180.254:44337/ using document root <none> and password file <none>
I20260808 15:03:17.381695 13870 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:17.381794 13870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:17.382102 13870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:17.384912 13870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "2e5f1219241d40549cb74ac5491f9a60"
format_stamp: "Formatted at 2026-08-08 15:03:17 on dist-test-slave-jlzn"
server_key: "c62b43d6dd0d1141e9babb9052b559c1"
server_key_iv: "086c35013fd2af086c2a39bf75c302bc"
server_key_version: "encryptionkey@0"
I20260808 15:03:17.385648 13870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "2e5f1219241d40549cb74ac5491f9a60"
format_stamp: "Formatted at 2026-08-08 15:03:17 on dist-test-slave-jlzn"
server_key: "c62b43d6dd0d1141e9babb9052b559c1"
server_key_iv: "086c35013fd2af086c2a39bf75c302bc"
server_key_version: "encryptionkey@0"
I20260808 15:03:17.390334 13870 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:03:17.393460 13893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:17.394635 13870 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:03:17.394791 13870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "2e5f1219241d40549cb74ac5491f9a60"
format_stamp: "Formatted at 2026-08-08 15:03:17 on dist-test-slave-jlzn"
server_key: "c62b43d6dd0d1141e9babb9052b559c1"
server_key_iv: "086c35013fd2af086c2a39bf75c302bc"
server_key_version: "encryptionkey@0"
I20260808 15:03:17.394982 13870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:17.413164 13870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:17.417068 13870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:17.417315 13870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:17.435849 13870 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:40235
I20260808 15:03:17.435902 13994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:40235 every 8 connection(s)
I20260808 15:03:17.437042 13870 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:17.440500 13995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:17.440675 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 13870
I20260808 15:03:17.440863 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:17.441489 2771 external_mini_cluster.cc:1428] Setting key ec0169fcf7273b6bc39091ba789f73eb
I20260808 15:03:17.453862 13995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: Bootstrap starting.
Aug 08 15:03:17 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:17.456682 13995 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:17.457927 13995 log.cc:824] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:17.461371 13995 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: No bootstrap required, opened a new log
I20260808 15:03:17.465004 13999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:17.443267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.2.180.254:40235)
0808 15:03:17.444009 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:17.444023 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:17.444075 (+ 52us) server_negotiation.cc:407] Connection header received
0808 15:03:17.446658 (+ 2583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:17.446684 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:17.447142 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:17.447568 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:17.448861 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:17.450753 (+ 1892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:17.452090 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:17.453086 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:17.453266 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:17.456940 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:17.456955 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:17.456973 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:17.457024 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:17.460791 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:17.461451 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:17.461459 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:17.461467 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:17.461688 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:17.462057 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:17.462063 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:17.462066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:17.462740 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:17.463031 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:17.463332 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 15:03:17.463727 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":298,"threads_started":1}
I20260808 15:03:17.470343 13995 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5f1219241d40549cb74ac5491f9a60" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40235 } }
I20260808 15:03:17.470640 13995 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:17.470724 13995 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5f1219241d40549cb74ac5491f9a60, State: Initialized, Role: FOLLOWER
I20260808 15:03:17.471208 13995 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5f1219241d40549cb74ac5491f9a60" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40235 } }
I20260808 15:03:17.471406 13995 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:17.471509 13995 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:17.471632 13995 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:17.473583 13995 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5f1219241d40549cb74ac5491f9a60" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40235 } }
I20260808 15:03:17.473994 13995 leader_election.cc:302] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e5f1219241d40549cb74ac5491f9a60; no voters:
I20260808 15:03:17.474464 13995 leader_election.cc:288] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:17.474568 14001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:17.475198 14001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [term 1 LEADER]: Becoming Leader. State: Replica: 2e5f1219241d40549cb74ac5491f9a60, State: Running, Role: LEADER
I20260808 15:03:17.475718 14001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5f1219241d40549cb74ac5491f9a60" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40235 } }
I20260808 15:03:17.476401 13995 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:17.478543 14003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e5f1219241d40549cb74ac5491f9a60. Latest consensus state: current_term: 1 leader_uuid: "2e5f1219241d40549cb74ac5491f9a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5f1219241d40549cb74ac5491f9a60" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40235 } } }
I20260808 15:03:17.478945 14003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:17.480221 14002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e5f1219241d40549cb74ac5491f9a60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e5f1219241d40549cb74ac5491f9a60" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40235 } } }
I20260808 15:03:17.480562 14002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:17.481014 14011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:17.485100 14011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:17.492377 14011 catalog_manager.cc:1347] Generated new cluster ID: 46a530051cec4bd981d57e96c08f94b0
I20260808 15:03:17.492478 14011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:17.509104 14011 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:17.510829 14011 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:17.521366 14011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: Generated new TSK 0
I20260808 15:03:17.522346 14011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab.
Entry for principal oryx/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab.
I20260808 15:03:17.570977 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40235
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab
--principal=oryx/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:17.885876 14030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:17.886123 14030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:17.886224 14030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:17.893028 14030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:17.893121 14030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:17.893206 14030 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:03:17.893282 14030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:17.902391 14030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab
--principal=oryx/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:17.903520 14030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:17.904737 14030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:17.916759 14036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:17.916841 14035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:17.916847 14038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:17.917727 14030 server_base.cc:1034] running on GCE node
I20260808 15:03:17.918229 14030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:17.919059 14030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:17.920292 14030 hybrid_clock.cc:648] HybridClock initialized: now 1786201397920221 us; error 25 us; skew 500 ppm
I20260808 15:03:17.920572 14030 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:17 dist-test-slave-jlzn krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201397, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:17.923503 14030 init.cc:377] Logged in from keytab as oryx/127.2.180.193@KRBTEST.COM (short username oryx)
I20260808 15:03:17.924839 14030 webserver.cc:460] Webserver started at http://127.2.180.193:44661/ using document root <none> and password file <none>
I20260808 15:03:17.925416 14030 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:17.925501 14030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:17.925768 14030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:17.928428 14030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "61ce5a7167af4b5b909db9eb814bbfd0"
format_stamp: "Formatted at 2026-08-08 15:03:17 on dist-test-slave-jlzn"
server_key: "d29dbd5763351d9b5a838d98f5ab6eb5"
server_key_iv: "a3ebf5cb1db4dace671b692075a124fa"
server_key_version: "encryptionkey@0"
I20260808 15:03:17.929142 14030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "61ce5a7167af4b5b909db9eb814bbfd0"
format_stamp: "Formatted at 2026-08-08 15:03:17 on dist-test-slave-jlzn"
server_key: "d29dbd5763351d9b5a838d98f5ab6eb5"
server_key_iv: "a3ebf5cb1db4dace671b692075a124fa"
server_key_version: "encryptionkey@0"
I20260808 15:03:17.933786 14030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:03:17.936883 14053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:17.938048 14030 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 15:03:17.938196 14030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "61ce5a7167af4b5b909db9eb814bbfd0"
format_stamp: "Formatted at 2026-08-08 15:03:17 on dist-test-slave-jlzn"
server_key: "d29dbd5763351d9b5a838d98f5ab6eb5"
server_key_iv: "a3ebf5cb1db4dace671b692075a124fa"
server_key_version: "encryptionkey@0"
I20260808 15:03:17.938387 14030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:17.979835 14030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:17.983939 14030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:17.984189 14030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:17.986605 14030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:17.986703 14030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:17.986838 14030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:17.986904 14030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:03:17 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201397, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:18.012606 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:17.987367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:46999 (local address 127.2.180.254:40235)
0808 15:03:17.988114 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:17.988118 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:17.988986 (+ 868us) server_negotiation.cc:407] Connection header received
0808 15:03:17.990964 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:17.990970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:17.991056 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:17.991157 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:17.993287 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:17.994269 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:17.996643 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:17.997221 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:17.997341 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.002634 (+ 5293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:18.002641 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:18.002646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:18.002690 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:18.006851 (+ 4161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.008367 (+ 1516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.008373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.008376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.008533 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.009145 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.009160 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.009163 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.009520 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:18.009673 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:18.011994 (+ 2321us) server_negotiation.cc:299] Negotiation successful
0808 15:03:18.012243 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":547,"threads_started":1}
I20260808 15:03:18.012907 14067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:17.988039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40235 (local address 127.2.180.193:46999)
0808 15:03:17.988836 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:17.988875 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:17.990713 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:17.991437 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:17.991451 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:17.992276 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:17.993111 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:17.993139 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:17.994422 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:17.994431 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:17.996506 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:17.996518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:17.997661 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:17.997668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:17.997925 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:17.998944 (+ 1019us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:17.999005 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:18.002376 (+ 3371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:18.006983 (+ 4607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.006997 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.007014 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.007751 (+ 737us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.008680 (+ 929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.008685 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.008693 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.008987 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.009675 (+ 688us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:18.009686 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:18.010205 (+ 519us) client_negotiation.cc:769] Sending connection context
0808 15:03:18.010556 (+ 351us) client_negotiation.cc:240] Negotiation successful
0808 15:03:18.011167 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":359,"threads_started":1}
I20260808 15:03:18.026764 14030 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:34337
I20260808 15:03:18.026823 14231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:34337 every 8 connection(s)
I20260808 15:03:18.028077 14030 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:18.029917 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 14030
I20260808 15:03:18.030117 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:03:18.030679 14232 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40235
I20260808 15:03:18.030900 14232 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:18.030910 2771 external_mini_cluster.cc:1428] Setting key f8b7977d491f37b170a9a7b2df81449f
I20260808 15:03:18.031778 14232 heartbeater.cc:511] Master 127.2.180.254:40235 requested a full tablet report, sending...
I20260808 15:03:18.033973 13919 ts_manager.cc:194] Registered new tserver with Master: 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337)
I20260808 15:03:18.035838 13919 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:46999
I20260808 15:03:18.040727 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.023956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:44505 (local address 127.2.180.254:40235)
0808 15:03:18.024141 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:18.024146 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:18.025090 (+ 944us) server_negotiation.cc:407] Connection header received
0808 15:03:18.025243 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:18.025247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:18.025304 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:18.025378 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:18.026658 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.027440 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.029351 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.029963 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.030077 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.032271 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:18.032277 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:18.032282 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:18.032317 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:18.038348 (+ 6031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.039085 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.039089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.039091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.039222 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.039557 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.039560 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.039562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.039835 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:18.039948 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:18.040354 (+ 406us) server_negotiation.cc:299] Negotiation successful
0808 15:03:18.040512 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:03:18.040908 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.023969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40235 (local address 127.2.180.193:44505)
0808 15:03:18.024994 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:18.025012 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:18.025137 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:18.025431 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:18.025436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:18.026067 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:18.026502 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.026517 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.027594 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.027600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.029217 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.029230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.030318 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.030325 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.030454 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.031065 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:18.031087 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:18.032080 (+ 993us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:18.038486 (+ 6406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.038490 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.038494 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.038962 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.039318 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.039322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.039325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.039470 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.039943 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:18.039948 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:18.040135 (+ 187us) client_negotiation.cc:769] Sending connection context
0808 15:03:18.040271 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 15:03:18.040532 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":873,"thread_start_us":727,"threads_started":1}
WARNING: no policy specified for oryx/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab.
Entry for principal oryx/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab.
I20260808 15:03:18.078763 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40235
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab
--principal=oryx/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:18.385661 14239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:18.385897 14239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:18.386024 14239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:18.393038 14239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:18.393131 14239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:18.393249 14239 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:03:18.393347 14239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:18.402599 14239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab
--principal=oryx/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:18.403775 14239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:18.405067 14239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:18.417129 14248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:18.417333 14249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:18.417637 14252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:18.418427 14239 server_base.cc:1034] running on GCE node
I20260808 15:03:18.418941 14239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:18.419720 14239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:18.420922 14239 hybrid_clock.cc:648] HybridClock initialized: now 1786201398420854 us; error 43 us; skew 500 ppm
I20260808 15:03:18.421192 14239 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:18 dist-test-slave-jlzn krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201398, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:18.424189 14239 init.cc:377] Logged in from keytab as oryx/127.2.180.194@KRBTEST.COM (short username oryx)
I20260808 15:03:18.425518 14239 webserver.cc:460] Webserver started at http://127.2.180.194:35889/ using document root <none> and password file <none>
I20260808 15:03:18.426127 14239 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:18.426234 14239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:18.426538 14239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:18.429422 14239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "4b07a78f42ca4880bbe2f912fddce8f5"
format_stamp: "Formatted at 2026-08-08 15:03:18 on dist-test-slave-jlzn"
server_key: "e39da5ea8e1c957e28eda0d1a73aeecc"
server_key_iv: "4107ce178f0885dc76bbc02e99946fc8"
server_key_version: "encryptionkey@0"
I20260808 15:03:18.430166 14239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "4b07a78f42ca4880bbe2f912fddce8f5"
format_stamp: "Formatted at 2026-08-08 15:03:18 on dist-test-slave-jlzn"
server_key: "e39da5ea8e1c957e28eda0d1a73aeecc"
server_key_iv: "4107ce178f0885dc76bbc02e99946fc8"
server_key_version: "encryptionkey@0"
I20260808 15:03:18.434846 14239 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:03:18.438380 14264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:18.439599 14239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:03:18.439738 14239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "4b07a78f42ca4880bbe2f912fddce8f5"
format_stamp: "Formatted at 2026-08-08 15:03:18 on dist-test-slave-jlzn"
server_key: "e39da5ea8e1c957e28eda0d1a73aeecc"
server_key_iv: "4107ce178f0885dc76bbc02e99946fc8"
server_key_version: "encryptionkey@0"
I20260808 15:03:18.439965 14239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:18.462528 14239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:18.466073 14239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:18.466320 14239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:18.468837 14239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:18.468928 14239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:18.469072 14239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:18.469161 14239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:03:18 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201398, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:18.489898 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.469175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:50067 (local address 127.2.180.254:40235)
0808 15:03:18.469376 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:18.469383 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:18.470748 (+ 1365us) server_negotiation.cc:407] Connection header received
0808 15:03:18.472839 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:18.472846 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:18.472934 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:18.473040 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:18.475017 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.476105 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.478160 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.478592 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.478704 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.483322 (+ 4618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:18.483327 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:18.483330 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:18.483363 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:18.486056 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.487055 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.487068 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.487072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.487225 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.487857 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.487862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.487866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.488234 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:18.488384 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:18.489328 (+ 944us) server_negotiation.cc:299] Negotiation successful
0808 15:03:18.489555 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:03:18.490810 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.469726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40235 (local address 127.2.180.194:50067)
0808 15:03:18.470567 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:18.470615 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:18.472539 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:18.473305 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:18.473312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:18.473955 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:18.474768 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.474806 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.476276 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.476282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.478023 (+ 1741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.478035 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.478813 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.478821 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.479034 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.479730 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:18.479765 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:18.483105 (+ 3340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:18.486214 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.486222 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.486240 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.486927 (+ 687us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.487346 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.487354 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.487359 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.487663 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.488459 (+ 796us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:18.488465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:18.488946 (+ 481us) client_negotiation.cc:769] Sending connection context
0808 15:03:18.489198 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 15:03:18.489726 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":338,"threads_started":1}
I20260808 15:03:18.509696 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.495893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:35447 (local address 127.2.180.254:40235)
0808 15:03:18.496075 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:18.496081 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:18.496748 (+ 667us) server_negotiation.cc:407] Connection header received
0808 15:03:18.497011 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:18.497018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:18.497109 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:18.497214 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:18.498402 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.499120 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.501144 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.501561 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.501655 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.503828 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:18.503836 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:18.503841 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:18.503878 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:18.506974 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.507782 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.507787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.507790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.507980 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.508433 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.508439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.508443 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.508757 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:18.508911 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:18.509318 (+ 407us) server_negotiation.cc:299] Negotiation successful
0808 15:03:18.509481 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:03:18.510048 14354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.495921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40235 (local address 127.2.180.194:35447)
0808 15:03:18.496646 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:18.496665 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:18.496858 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:18.497228 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:18.497231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:18.497689 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:18.498233 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.498252 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.499233 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.499239 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.500987 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.501004 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.501740 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.501747 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.501964 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.502564 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:18.502586 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:18.503615 (+ 1029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:18.507106 (+ 3491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.507112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.507116 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.507658 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.508132 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.508137 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.508140 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.508307 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.508892 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:18.508898 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:18.509101 (+ 203us) client_negotiation.cc:769] Sending connection context
0808 15:03:18.509251 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 15:03:18.509576 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":406,"threads_started":1}
I20260808 15:03:18.522675 14239 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:45859
I20260808 15:03:18.522717 14441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:45859 every 8 connection(s)
I20260808 15:03:18.523794 14239 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:18.525790 14442 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40235
I20260808 15:03:18.526021 14442 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:18.525993 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 14239
I20260808 15:03:18.526157 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:03:18.526717 2771 external_mini_cluster.cc:1428] Setting key c9b78fc0a436bf5402c78afb8d10c4e6
I20260808 15:03:18.526789 14442 heartbeater.cc:511] Master 127.2.180.254:40235 requested a full tablet report, sending...
I20260808 15:03:18.528252 13916 ts_manager.cc:194] Registered new tserver with Master: 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859)
I20260808 15:03:18.529100 13916 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:50067
WARNING: no policy specified for oryx/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab.
Entry for principal oryx/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab.
I20260808 15:03:18.571712 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40235
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab
--principal=oryx/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:18.881568 14447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:18.881821 14447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:18.881932 14447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:18.888875 14447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:18.888968 14447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:18.889060 14447 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 15:03:18.889138 14447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:18.898334 14447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab
--principal=oryx/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.2.180.254:40235
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:18.899544 14447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:18.900799 14447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:18.912972 14456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:18.913196 14455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:18.913010 14458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:18.913967 14447 server_base.cc:1034] running on GCE node
I20260808 15:03:18.914499 14447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:18.915343 14447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:18.916553 14447 hybrid_clock.cc:648] HybridClock initialized: now 1786201398916486 us; error 35 us; skew 500 ppm
I20260808 15:03:18.916812 14447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:18 dist-test-slave-jlzn krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201398, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:18.919818 14447 init.cc:377] Logged in from keytab as oryx/127.2.180.195@KRBTEST.COM (short username oryx)
I20260808 15:03:18.921146 14447 webserver.cc:460] Webserver started at http://127.2.180.195:45135/ using document root <none> and password file <none>
I20260808 15:03:18.921766 14447 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:18.921864 14447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:18.922158 14447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:18.924842 14447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "0c6e0d9698c64d05b404c6ea5ee468f5"
format_stamp: "Formatted at 2026-08-08 15:03:18 on dist-test-slave-jlzn"
server_key: "dac5e20ed5f49c72aadf98e54b89903b"
server_key_iv: "7a9adacdca17cffb2ed4e580c44a3a04"
server_key_version: "encryptionkey@0"
I20260808 15:03:18.925578 14447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "0c6e0d9698c64d05b404c6ea5ee468f5"
format_stamp: "Formatted at 2026-08-08 15:03:18 on dist-test-slave-jlzn"
server_key: "dac5e20ed5f49c72aadf98e54b89903b"
server_key_iv: "7a9adacdca17cffb2ed4e580c44a3a04"
server_key_version: "encryptionkey@0"
I20260808 15:03:18.930330 14447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 15:03:18.933694 14470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:18.934897 14447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 15:03:18.935043 14447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "0c6e0d9698c64d05b404c6ea5ee468f5"
format_stamp: "Formatted at 2026-08-08 15:03:18 on dist-test-slave-jlzn"
server_key: "dac5e20ed5f49c72aadf98e54b89903b"
server_key_iv: "7a9adacdca17cffb2ed4e580c44a3a04"
server_key_version: "encryptionkey@0"
I20260808 15:03:18.935235 14447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:18.953742 14447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:18.957281 14447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:18.957513 14447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:18.960065 14447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:18.960170 14447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:18.960348 14447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:18.960422 14447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:03:18 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201398, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:18.983908 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.961043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40235 (local address 127.2.180.195:36151)
0808 15:03:18.961822 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:18.961864 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:18.963653 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:18.964384 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:18.964392 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:18.965084 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:18.965917 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.965948 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.967478 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.967484 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.969583 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.969601 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.970700 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.970707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.971049 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.971879 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:18.971913 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:18.974965 (+ 3052us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:18.978658 (+ 3693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.978666 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.978681 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.979305 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.979767 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:18.979771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:18.979775 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:18.980075 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:18.980898 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:18.980906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:18.981390 (+ 484us) client_negotiation.cc:769] Sending connection context
0808 15:03:18.981716 (+ 326us) client_negotiation.cc:240] Negotiation successful
0808 15:03:18.982273 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":317,"threads_started":1}
I20260808 15:03:18.985344 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.960461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:36151 (local address 127.2.180.254:40235)
0808 15:03:18.960651 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:18.960658 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:18.961982 (+ 1324us) server_negotiation.cc:407] Connection header received
0808 15:03:18.963940 (+ 1958us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:18.963946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:18.964034 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:18.964129 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:18.966111 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.967252 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.969745 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.970420 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.970552 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.975221 (+ 4669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:18.975228 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:18.975232 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:18.975264 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:18.978458 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.979436 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.979442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.979446 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.979621 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:18.980267 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:18.980271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:18.980274 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:18.980653 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:18.980803 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:18.984733 (+ 3930us) server_negotiation.cc:299] Negotiation successful
0808 15:03:18.984976 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:03:18.999305 14447 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:45383
I20260808 15:03:18.999373 14666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:45383 every 8 connection(s)
I20260808 15:03:19.000733 14447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:19.002627 14667 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40235
I20260808 15:03:19.002831 14667 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:19.003409 14667 heartbeater.cc:511] Master 127.2.180.254:40235 requested a full tablet report, sending...
I20260808 15:03:19.004712 13916 ts_manager.cc:194] Registered new tserver with Master: 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383)
I20260808 15:03:19.005512 13916 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:36151
I20260808 15:03:19.005985 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.992035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:54725 (local address 127.2.180.254:40235)
0808 15:03:18.992200 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:18.992206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:18.992856 (+ 650us) server_negotiation.cc:407] Connection header received
0808 15:03:18.993160 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:18.993165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:18.993243 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:18.993311 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:18.994717 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.995679 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.997677 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.998182 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.998300 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:19.000498 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:19.000502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:19.000505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:19.000534 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:19.003460 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:19.004121 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:19.004127 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:19.004130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:19.004253 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:19.004606 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:19.004612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:19.004616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:19.004910 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:19.005058 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:19.005449 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 15:03:19.005683 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:03:19.006007 14634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:18.992075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40235 (local address 127.2.180.195:54725)
0808 15:03:18.992755 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:18.992775 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:18.992962 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:18.993512 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:18.993518 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:18.994112 (+ 594us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:18.994532 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.994548 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.995904 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.995909 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.997544 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:18.997558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:18.998409 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:18.998416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:18.998610 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:18.999290 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:18.999312 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:19.000319 (+ 1007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:19.003570 (+ 3251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:19.003575 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:19.003579 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:19.004006 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:19.004357 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:19.004362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:19.004366 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:19.004485 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:19.005024 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:19.005028 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:19.005201 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 15:03:19.005308 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 15:03:19.005576 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":353,"threads_started":1}
I20260808 15:03:19.008169 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 14447
I20260808 15:03:19.008322 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:03:19.008780 2771 external_mini_cluster.cc:1428] Setting key f0efc824ffdeb65880f5b2cf61a3ba11
I20260808 15:03:19.011320 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:03:19.038998 14232 heartbeater.cc:503] Master 127.2.180.254:40235 was elected leader, sending a full tablet report...
Aug 08 15:03:19 dist-test-slave-jlzn krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 15:03:19 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:19 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:19.531368 14442 heartbeater.cc:503] Master 127.2.180.254:40235 was elected leader, sending a full tablet report...
I20260808 15:03:20.007865 14667 heartbeater.cc:503] Master 127.2.180.254:40235 was elected leader, sending a full tablet report...
W20260808 15:03:20.179168 14682 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.122s user 0.011s sys 0.003s
W20260808 15:03:20.180434 14065 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:19.051888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.2.180.254:40235)
0808 15:03:19.052096 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:19.052102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:19.052274 (+ 172us) server_negotiation.cc:407] Connection header received
0808 15:03:19.052448 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:19.052452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:19.052526 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:19.052606 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:19.053885 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:19.054605 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:19.055850 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:19.056267 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:19.056377 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.180124 (+1123747us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35952: BlockingRecv error: recv got EOF from 127.0.0.1:35952 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 15:03:20 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201399, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:20.197367 14065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.184271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.2.180.254:40235)
0808 15:03:20.184507 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.184513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.184795 (+ 282us) server_negotiation.cc:407] Connection header received
0808 15:03:20.184940 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.184944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.185034 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.185121 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:20.186320 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.187040 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.188375 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.188790 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.188879 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.192609 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:20.192614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:20.192617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:20.192651 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:20.195264 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:20.195848 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:20.195853 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:20.195856 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:20.195981 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:20.196327 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:20.196331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:20.196333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:20.196618 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:20.196848 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.196991 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.197147 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:03:20.201207 13916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:03:20.207960 13916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:03:20.232045 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.221577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.2.180.195:45383)
0808 15:03:20.224273 (+ 2696us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.224283 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.225478 (+ 1195us) server_negotiation.cc:407] Connection header received
0808 15:03:20.225657 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.225663 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.225818 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.225996 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:20.226469 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.227925 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.229964 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.230970 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.231092 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.231301 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.231576 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.231743 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2519,"thread_start_us":2394,"threads_started":1}
I20260808 15:03:20.232105 14702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.221293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:45383 (local address 127.0.0.1:43586)
0808 15:03:20.225371 (+ 4078us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:20.225384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:20.225502 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:20.225980 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:20.225985 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:20.226009 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:20.226333 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.226342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.228084 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.228090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.229819 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.229832 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.231171 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.231178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.231366 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.231404 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 15:03:20.231517 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 15:03:20.231702 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3968,"mutex_wait_us":5,"thread_start_us":271,"threads_started":1}
I20260808 15:03:20.233481 14704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.219116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34337 (local address 127.0.0.1:34354)
0808 15:03:20.220356 (+ 1240us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:20.220398 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:20.220583 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:20.225361 (+ 4778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:20.225369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:20.225418 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:20.225836 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.225848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.227592 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.227597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.230100 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.230116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.232735 (+ 2619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.232742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.232896 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.232929 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:03:20.232994 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 15:03:20.233115 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":959,"mutex_wait_us":86,"thread_start_us":5998,"threads_started":1}
I20260808 15:03:20.234061 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.219370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34354 (local address 127.2.180.193:34337)
0808 15:03:20.224695 (+ 5325us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.224703 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.224735 (+ 32us) server_negotiation.cc:407] Connection header received
0808 15:03:20.224837 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.224845 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.225094 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.226057 (+ 963us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:20.226288 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.227431 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.230286 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.232492 (+ 2206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.233346 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.233549 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.233614 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.233783 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5179,"thread_start_us":5038,"threads_started":1}
I20260808 15:03:20.234273 14710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.219591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:45859 (local address 127.0.0.1:38578)
0808 15:03:20.221799 (+ 2208us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:20.221817 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:20.221927 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:20.227380 (+ 5453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:20.227385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:20.227410 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:20.227857 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.227872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.229832 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.229837 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.231763 (+ 1926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.231778 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.233604 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.233611 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.233764 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.233795 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:03:20.233859 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 15:03:20.233968 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2095,"mutex_wait_us":2,"thread_start_us":789,"threads_started":1}
I20260808 15:03:20.235067 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.220207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38578 (local address 127.2.180.194:45859)
0808 15:03:20.226870 (+ 6663us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.226878 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.226902 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:03:20.226989 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.226996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.227190 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.227368 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:20.228194 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.229685 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.231983 (+ 2298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.233385 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.234375 (+ 990us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.234590 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.234675 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.234791 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6506,"thread_start_us":6372,"threads_started":1}
I20260808 15:03:20.235899 14549 tablet_service.cc:1467] Processing CreateTablet for tablet 5d325a0ee8f24d3589a36bd78dcfb34e (DEFAULT_TABLE table=test-table [id=96f061d69910434c86b77671b8856d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:20.237612 14549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d325a0ee8f24d3589a36bd78dcfb34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:20.237376 14128 tablet_service.cc:1467] Processing CreateTablet for tablet 5d325a0ee8f24d3589a36bd78dcfb34e (DEFAULT_TABLE table=test-table [id=96f061d69910434c86b77671b8856d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:20.238534 14128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d325a0ee8f24d3589a36bd78dcfb34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:20.238468 14337 tablet_service.cc:1467] Processing CreateTablet for tablet 5d325a0ee8f24d3589a36bd78dcfb34e (DEFAULT_TABLE table=test-table [id=96f061d69910434c86b77671b8856d05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:20.239732 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5d325a0ee8f24d3589a36bd78dcfb34e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:20.249189 14712 tablet_bootstrap.cc:492] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Bootstrap starting.
I20260808 15:03:20.251693 14713 tablet_bootstrap.cc:492] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Bootstrap starting.
I20260808 15:03:20.253000 14714 tablet_bootstrap.cc:492] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Bootstrap starting.
I20260808 15:03:20.253501 14712 tablet_bootstrap.cc:654] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:20.254801 14712 log.cc:824] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:20.255041 14713 tablet_bootstrap.cc:654] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:20.255864 14714 tablet_bootstrap.cc:654] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:20.256145 14713 log.cc:824] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:20.256919 14714 log.cc:824] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:20.257712 14712 tablet_bootstrap.cc:492] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: No bootstrap required, opened a new log
I20260808 15:03:20.257987 14712 ts_tablet_manager.cc:1397] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 15:03:20.259274 14713 tablet_bootstrap.cc:492] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: No bootstrap required, opened a new log
I20260808 15:03:20.259461 14714 tablet_bootstrap.cc:492] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: No bootstrap required, opened a new log
I20260808 15:03:20.259531 14713 ts_tablet_manager.cc:1397] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 15:03:20.259709 14714 ts_tablet_manager.cc:1397] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 15:03:20.266072 14714 raft_consensus.cc:348] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.266389 14714 raft_consensus.cc:374] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:20.266481 14714 raft_consensus.cc:729] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b07a78f42ca4880bbe2f912fddce8f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:20.267000 14714 consensus_queue.cc:260] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.268297 14712 raft_consensus.cc:348] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.268584 14714 ts_tablet_manager.cc:1428] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent starting tablet: real 0.009s user 0.000s sys 0.010s
I20260808 15:03:20.268745 14712 raft_consensus.cc:374] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:20.268848 14712 raft_consensus.cc:729] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ce5a7167af4b5b909db9eb814bbfd0, State: Initialized, Role: FOLLOWER
I20260808 15:03:20.269647 14713 raft_consensus.cc:348] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.269810 14712 consensus_queue.cc:260] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.270097 14713 raft_consensus.cc:374] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:20.270200 14713 raft_consensus.cc:729] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6e0d9698c64d05b404c6ea5ee468f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:20.270774 14713 consensus_queue.cc:260] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.272496 14713 ts_tablet_manager.cc:1428] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260808 15:03:20.274427 14712 ts_tablet_manager.cc:1428] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
W20260808 15:03:20.276115 14443 tablet.cc:2369] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:03:20.281139 14233 tablet.cc:2369] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:03:20.503014 14668 tablet.cc:2369] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:03:20.553838 14723 raft_consensus.cc:482] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:20.554129 14723 raft_consensus.cc:504] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.555861 14723 leader_election.cc:288] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337), 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859)
I20260808 15:03:20.560370 14722 raft_consensus.cc:482] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:20.560642 14722 raft_consensus.cc:504] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.562245 14722 leader_election.cc:288] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859), 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383)
I20260808 15:03:20.564885 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.556284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:45829 (local address 127.2.180.194:45859)
0808 15:03:20.556706 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.556712 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.557053 (+ 341us) server_negotiation.cc:407] Connection header received
0808 15:03:20.557277 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.557282 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.557365 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.557472 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:20.558213 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.559319 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.562310 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.563573 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.563685 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.563757 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.564498 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.564633 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260808 15:03:20.564884 14703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.556283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:45797 (local address 127.2.180.193:34337)
0808 15:03:20.556508 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.556515 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.557188 (+ 673us) server_negotiation.cc:407] Connection header received
0808 15:03:20.557325 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.557330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.557399 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.557478 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:20.558213 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.559327 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.562310 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.563548 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.563694 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.563757 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.564498 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.564633 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:03:20.565635 14725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.556268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34337 (local address 127.2.180.195:45797)
0808 15:03:20.557088 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:20.557102 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:20.557210 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:20.557496 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:20.557501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:20.557591 (+ 90us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:20.558036 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.558050 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.559445 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.559449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.562121 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.562137 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.564009 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.564014 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.564282 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.564341 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 15:03:20.565106 (+ 765us) client_negotiation.cc:240] Negotiation successful
0808 15:03:20.565236 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"mutex_wait_us":286,"spinlock_wait_cycles":23424,"thread_start_us":577,"threads_started":1}
I20260808 15:03:20.565941 14724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.556151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:45859 (local address 127.2.180.195:45829)
0808 15:03:20.556916 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:20.556939 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:20.557114 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:20.557496 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:20.557501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:20.557530 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:20.558049 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.558059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.559442 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.559447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.562121 (+ 2674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.562137 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.563992 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.563999 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.564278 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.564344 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 15:03:20.565442 (+ 1098us) client_negotiation.cc:240] Negotiation successful
0808 15:03:20.565567 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"mutex_wait_us":240,"thread_start_us":398,"threads_started":1}
I20260808 15:03:20.566968 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d325a0ee8f24d3589a36bd78dcfb34e" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" is_pre_election: true
I20260808 15:03:20.566967 14158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d325a0ee8f24d3589a36bd78dcfb34e" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" is_pre_election: true
I20260808 15:03:20.567456 14158 raft_consensus.cc:2454] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 0.
I20260808 15:03:20.567513 14374 raft_consensus.cc:2454] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 0.
I20260808 15:03:20.568225 14473 leader_election.cc:302] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6e0d9698c64d05b404c6ea5ee468f5, 61ce5a7167af4b5b909db9eb814bbfd0; no voters:
I20260808 15:03:20.568791 14723 raft_consensus.cc:2781] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:03:20.568893 14723 raft_consensus.cc:482] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:03:20.568951 14723 raft_consensus.cc:3037] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:20.571022 14723 raft_consensus.cc:504] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.571588 14723 leader_election.cc:288] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 election: Requested vote from peers 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337), 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859)
I20260808 15:03:20.571998 14158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d325a0ee8f24d3589a36bd78dcfb34e" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ce5a7167af4b5b909db9eb814bbfd0"
I20260808 15:03:20.572175 14158 raft_consensus.cc:3037] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:20.574609 14158 raft_consensus.cc:2454] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 1.
I20260808 15:03:20.575284 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d325a0ee8f24d3589a36bd78dcfb34e" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5"
I20260808 15:03:20.575416 14473 leader_election.cc:302] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6e0d9698c64d05b404c6ea5ee468f5, 61ce5a7167af4b5b909db9eb814bbfd0; no voters:
I20260808 15:03:20.575501 14374 raft_consensus.cc:3037] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:20.578589 14723 raft_consensus.cc:2781] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:20.578676 14374 raft_consensus.cc:2454] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 1.
I20260808 15:03:20.582449 14706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.562825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:38791 (local address 127.2.180.194:45859)
0808 15:03:20.565406 (+ 2581us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.565411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.574510 (+ 9099us) server_negotiation.cc:407] Connection header received
0808 15:03:20.574718 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.574723 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.574797 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.574881 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:20.576320 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.577538 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.580378 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.581369 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.581473 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.581538 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.582027 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.582147 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2480,"thread_start_us":14969,"threads_started":1}
I20260808 15:03:20.582489 14726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.562419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:45859 (local address 127.2.180.193:38791)
0808 15:03:20.574386 (+ 11967us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:20.574407 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:20.574561 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:20.575743 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:20.575749 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:20.575778 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:20.576131 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.576142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.577709 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.577715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.580213 (+ 2498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.580228 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.581721 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.581727 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.581871 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.581894 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:03:20.581965 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 15:03:20.582102 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11824,"mutex_wait_us":5,"thread_start_us":12239,"threads_started":1}
I20260808 15:03:20.583185 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d325a0ee8f24d3589a36bd78dcfb34e" candidate_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" is_pre_election: true
I20260808 15:03:20.583509 14374 raft_consensus.cc:2379] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 61ce5a7167af4b5b909db9eb814bbfd0 in current term 1: Already voted for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in this term.
I20260808 15:03:20.583560 14723 raft_consensus.cc:686] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 LEADER]: Becoming Leader. State: Replica: 0c6e0d9698c64d05b404c6ea5ee468f5, State: Running, Role: LEADER
I20260808 15:03:20.584383 14723 consensus_queue.cc:237] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.585397 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.575140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:36677 (local address 127.2.180.195:45383)
0808 15:03:20.578384 (+ 3244us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.578390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.578440 (+ 50us) server_negotiation.cc:407] Connection header received
0808 15:03:20.578697 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.578702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.578790 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.578854 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:20.579705 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.580846 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.583341 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.584571 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.584714 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.584781 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.585045 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.585180 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3128}
I20260808 15:03:20.585435 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.575312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:45383 (local address 127.2.180.193:36677)
0808 15:03:20.578350 (+ 3038us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:20.578368 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:20.578540 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:20.579058 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:20.579064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:20.579091 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:20.579509 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.579521 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.581052 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.581058 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.583156 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:20.583174 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.584770 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.584776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:20.584908 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.584931 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:03:20.585009 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 15:03:20.585131 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2910,"thread_start_us":2728,"threads_started":1}
I20260808 15:03:20.586272 14583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5d325a0ee8f24d3589a36bd78dcfb34e" candidate_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" is_pre_election: true
I20260808 15:03:20.587340 14059 leader_election.cc:302] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 61ce5a7167af4b5b909db9eb814bbfd0; no voters: 0c6e0d9698c64d05b404c6ea5ee468f5, 4b07a78f42ca4880bbe2f912fddce8f5
I20260808 15:03:20.587718 14722 raft_consensus.cc:2726] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 15:03:20.589425 13916 catalog_manager.cc:5495] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } health_report { overall_health: HEALTHY } } }
I20260808 15:03:20.679840 14711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:20.675072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.2.180.195:45383)
0808 15:03:20.675244 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:20.675249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:20.675265 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:03:20.675352 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:20.675356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:20.675434 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:20.675554 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:20.675846 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.676649 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.677927 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:20.678314 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:20.678400 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:20.678735 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:20.679282 (+ 547us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:20.679389 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:20.679444 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 15:03:20.679521 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:03:20.687517 14158 raft_consensus.cc:1261] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Refusing update from remote peer 0c6e0d9698c64d05b404c6ea5ee468f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:20.687505 14374 raft_consensus.cc:1261] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Refusing update from remote peer 0c6e0d9698c64d05b404c6ea5ee468f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:20.688405 14729 consensus_queue.cc:1035] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:20.688723 14723 consensus_queue.cc:1035] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:20.698357 14736 mvcc.cc:205] Tried to move back new op lower bound from 7316280937207042048 to 7316280936798011392. Current Snapshot: MvccSnapshot[applied={T|T < 7316280937207042048}]
I20260808 15:03:20.702440 14737 mvcc.cc:205] Tried to move back new op lower bound from 7316280937207042048 to 7316280936798011392. Current Snapshot: MvccSnapshot[applied={T|T < 7316280937207042048}]
I20260808 15:03:20.702744 14738 mvcc.cc:205] Tried to move back new op lower bound from 7316280937207042048 to 7316280936798011392. Current Snapshot: MvccSnapshot[applied={T|T < 7316280937207042048 or (T in {7316280937207042048})}]
I20260808 15:03:20.712774 13916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:20.713008 13916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:20.716300 13916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: Sending DeleteTablet for 3 replicas of tablet 5d325a0ee8f24d3589a36bd78dcfb34e
I20260808 15:03:20.717310 14128 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d325a0ee8f24d3589a36bd78dcfb34e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:20 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:34354
I20260808 15:03:20.717643 14337 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d325a0ee8f24d3589a36bd78dcfb34e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:20 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:38578
I20260808 15:03:20.717830 14549 tablet_service.cc:1514] Processing DeleteTablet for tablet 5d325a0ee8f24d3589a36bd78dcfb34e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:20 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:43586
I20260808 15:03:20.722378 13916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:03:20.723692 13916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:03:20.724243 14751 tablet_replica.cc:330] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: stopping tablet replica
I20260808 15:03:20.724720 14751 raft_consensus.cc:2229] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:03:20.725508 14751 raft_consensus.cc:2258] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:20.727481 14751 ts_tablet_manager.cc:1905] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:20.731251 14549 tablet_service.cc:1467] Processing CreateTablet for tablet 2372428266004959852ea451c7533057 (DEFAULT_TABLE table=test-table [id=358d32fb304b4b208a96dc81a8ecb5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:20.731334 14336 tablet_service.cc:1467] Processing CreateTablet for tablet 2372428266004959852ea451c7533057 (DEFAULT_TABLE table=test-table [id=358d32fb304b4b208a96dc81a8ecb5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:20.731736 14549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2372428266004959852ea451c7533057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:20.731954 14336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2372428266004959852ea451c7533057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:20.735389 14751 ts_tablet_manager.cc:1918] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:20.735556 14751 log.cc:1196] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/5d325a0ee8f24d3589a36bd78dcfb34e
I20260808 15:03:20.736217 14749 tablet_replica.cc:330] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: stopping tablet replica
I20260808 15:03:20.736657 14751 ts_tablet_manager.cc:1939] T 5d325a0ee8f24d3589a36bd78dcfb34e P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting consensus metadata
I20260808 15:03:20.739336 14714 tablet_bootstrap.cc:492] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: Bootstrap starting.
I20260808 15:03:20.739789 14750 tablet_replica.cc:330] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: stopping tablet replica
I20260808 15:03:20.740260 14750 raft_consensus.cc:2229] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:20.740674 14714 tablet_bootstrap.cc:654] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:20.740767 14750 raft_consensus.cc:2258] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:20.737063 14127 tablet_service.cc:1467] Processing CreateTablet for tablet 2372428266004959852ea451c7533057 (DEFAULT_TABLE table=test-table [id=358d32fb304b4b208a96dc81a8ecb5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:20.741603 14127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2372428266004959852ea451c7533057. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:20.742744 14714 tablet_bootstrap.cc:492] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: No bootstrap required, opened a new log
I20260808 15:03:20.742880 14714 ts_tablet_manager.cc:1397] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 15:03:20.743506 14714 raft_consensus.cc:348] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.743712 14714 raft_consensus.cc:374] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:20.743778 14714 raft_consensus.cc:729] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b07a78f42ca4880bbe2f912fddce8f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:20.744268 14749 raft_consensus.cc:2229] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:20.744215 14714 consensus_queue.cc:260] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.746433 14749 raft_consensus.cc:2258] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:20.747184 13899 catalog_manager.cc:4841] TS 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383): tablet 5d325a0ee8f24d3589a36bd78dcfb34e (table test-table [id=96f061d69910434c86b77671b8856d05]) successfully deleted
I20260808 15:03:20.747772 14714 ts_tablet_manager.cc:1428] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20260808 15:03:20.748303 14712 tablet_bootstrap.cc:492] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: Bootstrap starting.
I20260808 15:03:20.748704 14749 ts_tablet_manager.cc:1905] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:20.750065 14712 tablet_bootstrap.cc:654] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:20.750906 14713 tablet_bootstrap.cc:492] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: Bootstrap starting.
I20260808 15:03:20.752468 14713 tablet_bootstrap.cc:654] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:20.753363 14750 ts_tablet_manager.cc:1905] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:20.759634 14713 tablet_bootstrap.cc:492] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: No bootstrap required, opened a new log
I20260808 15:03:20.760330 14713 ts_tablet_manager.cc:1397] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 15:03:20.760715 14712 tablet_bootstrap.cc:492] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: No bootstrap required, opened a new log
I20260808 15:03:20.760923 14712 ts_tablet_manager.cc:1397] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260808 15:03:20.761193 14713 raft_consensus.cc:348] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.761432 14713 raft_consensus.cc:374] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:20.761544 14713 raft_consensus.cc:729] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6e0d9698c64d05b404c6ea5ee468f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:20.761809 14713 consensus_queue.cc:260] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.762221 14712 raft_consensus.cc:348] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.762498 14712 raft_consensus.cc:374] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:20.762989 14713 ts_tablet_manager.cc:1428] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:03:20.766470 14712 raft_consensus.cc:729] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ce5a7167af4b5b909db9eb814bbfd0, State: Initialized, Role: FOLLOWER
I20260808 15:03:20.766865 14712 consensus_queue.cc:260] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:20.770005 14750 ts_tablet_manager.cc:1918] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:20.770159 14750 log.cc:1196] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/5d325a0ee8f24d3589a36bd78dcfb34e
I20260808 15:03:20.770682 14750 ts_tablet_manager.cc:1939] T 5d325a0ee8f24d3589a36bd78dcfb34e P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting consensus metadata
I20260808 15:03:20.770808 14712 ts_tablet_manager.cc:1428] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s
I20260808 15:03:20.774526 13899 catalog_manager.cc:4841] TS 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859): tablet 5d325a0ee8f24d3589a36bd78dcfb34e (table test-table [id=96f061d69910434c86b77671b8856d05]) successfully deleted
I20260808 15:03:20.775275 14749 ts_tablet_manager.cc:1918] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:20.775415 14749 log.cc:1196] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/5d325a0ee8f24d3589a36bd78dcfb34e
I20260808 15:03:20.775933 14749 ts_tablet_manager.cc:1939] T 5d325a0ee8f24d3589a36bd78dcfb34e P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting consensus metadata
I20260808 15:03:20.777535 13895 catalog_manager.cc:4841] TS 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337): tablet 5d325a0ee8f24d3589a36bd78dcfb34e (table test-table [id=96f061d69910434c86b77671b8856d05]) successfully deleted
I20260808 15:03:21.199249 14723 raft_consensus.cc:482] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:21.199450 14723 raft_consensus.cc:504] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.200122 14723 leader_election.cc:288] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337), 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859)
I20260808 15:03:21.200703 14158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2372428266004959852ea451c7533057" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" is_pre_election: true
I20260808 15:03:21.200736 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2372428266004959852ea451c7533057" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" is_pre_election: true
I20260808 15:03:21.200924 14158 raft_consensus.cc:2454] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 0.
I20260808 15:03:21.200974 14374 raft_consensus.cc:2454] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 0.
I20260808 15:03:21.201344 14473 leader_election.cc:302] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6e0d9698c64d05b404c6ea5ee468f5, 61ce5a7167af4b5b909db9eb814bbfd0; no voters:
I20260808 15:03:21.201601 14723 raft_consensus.cc:2781] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:03:21.201722 14723 raft_consensus.cc:482] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:03:21.201817 14723 raft_consensus.cc:3037] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.203890 14723 raft_consensus.cc:504] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.204382 14723 leader_election.cc:288] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 election: Requested vote from peers 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337), 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859)
I20260808 15:03:21.204787 14158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2372428266004959852ea451c7533057" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ce5a7167af4b5b909db9eb814bbfd0"
I20260808 15:03:21.204869 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2372428266004959852ea451c7533057" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5"
I20260808 15:03:21.204993 14158 raft_consensus.cc:3037] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.205045 14374 raft_consensus.cc:3037] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.207828 14158 raft_consensus.cc:2454] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 1.
I20260808 15:03:21.207828 14374 raft_consensus.cc:2454] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 1.
I20260808 15:03:21.208202 14473 leader_election.cc:302] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0c6e0d9698c64d05b404c6ea5ee468f5, 4b07a78f42ca4880bbe2f912fddce8f5, 61ce5a7167af4b5b909db9eb814bbfd0; no voters:
I20260808 15:03:21.208472 14723 raft_consensus.cc:2781] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:21.208782 14723 raft_consensus.cc:686] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 LEADER]: Becoming Leader. State: Replica: 0c6e0d9698c64d05b404c6ea5ee468f5, State: Running, Role: LEADER
I20260808 15:03:21.209033 14723 consensus_queue.cc:237] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.211361 13916 catalog_manager.cc:5495] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } health_report { overall_health: HEALTHY } } }
Aug 08 15:03:21 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201397, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.254@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:21.265324 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:21.245822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40235 (local address 127.0.0.1:35972)
0808 15:03:21.246491 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:21.246509 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:21.246656 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:21.246946 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:21.246950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:21.247581 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:21.247786 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:21.247795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.248863 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.248868 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:21.249815 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:21.249827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.250522 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.250528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:21.250715 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.254018 (+ 3303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:21.254043 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:21.259116 (+ 5073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:21.262319 (+ 3203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:21.262334 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:21.262343 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:21.263040 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:21.263411 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:21.263416 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:21.263419 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:21.263594 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:21.264345 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:21.264355 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:21.264555 (+ 200us) client_negotiation.cc:769] Sending connection context
0808 15:03:21.264719 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 15:03:21.264969 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":407,"threads_started":1}
I20260808 15:03:21.266115 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:21.245808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35972 (local address 127.2.180.254:40235)
0808 15:03:21.246474 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:21.246480 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:21.246576 (+ 96us) server_negotiation.cc:407] Connection header received
0808 15:03:21.246765 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:21.246769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:21.246822 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:21.246909 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:21.247959 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.248741 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.249944 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.250372 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.250477 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.259344 (+ 8867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:21.259352 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:21.259357 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:21.259395 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:21.262180 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:21.263159 (+ 979us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:21.263163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:21.263167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:21.263301 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:21.263712 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:21.263716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:21.263719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:21.264124 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:21.264273 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:21.265591 (+ 1318us) server_negotiation.cc:299] Negotiation successful
0808 15:03:21.265788 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":274,"threads_started":1}
I20260808 15:03:21.272629 13916 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:35972:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 15:03:21.273696 13916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:03:21.280278 14128 tablet_service.cc:1467] Processing CreateTablet for tablet 9f31ea4807d44aa5a1c8e2015378b5ac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0369001f046b48439d6f1eb0a18dc136]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:03:21.280871 14128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f31ea4807d44aa5a1c8e2015378b5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.280727 14337 tablet_service.cc:1467] Processing CreateTablet for tablet 9f31ea4807d44aa5a1c8e2015378b5ac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0369001f046b48439d6f1eb0a18dc136]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:03:21.280787 14549 tablet_service.cc:1467] Processing CreateTablet for tablet 9f31ea4807d44aa5a1c8e2015378b5ac (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0369001f046b48439d6f1eb0a18dc136]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 15:03:21.281239 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f31ea4807d44aa5a1c8e2015378b5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.281358 14549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9f31ea4807d44aa5a1c8e2015378b5ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.287331 14771 tablet_bootstrap.cc:492] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: Bootstrap starting.
I20260808 15:03:21.288939 14771 tablet_bootstrap.cc:654] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.290828 14771 tablet_bootstrap.cc:492] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: No bootstrap required, opened a new log
I20260808 15:03:21.290964 14771 ts_tablet_manager.cc:1397] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 15:03:21.291544 14771 raft_consensus.cc:348] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.291780 14771 raft_consensus.cc:374] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.291901 14771 raft_consensus.cc:729] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6e0d9698c64d05b404c6ea5ee468f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.292188 14771 consensus_queue.cc:260] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.292541 14723 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.292856 14723 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:03:21.293879 14777 tablet_bootstrap.cc:492] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: Bootstrap starting.
I20260808 15:03:21.295257 14771 ts_tablet_manager.cc:1428] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260808 15:03:21.295588 14777 tablet_bootstrap.cc:654] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.296206 14778 tablet_bootstrap.cc:492] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: Bootstrap starting.
I20260808 15:03:21.298092 14777 tablet_bootstrap.cc:492] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: No bootstrap required, opened a new log
I20260808 15:03:21.298285 14777 ts_tablet_manager.cc:1397] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20260808 15:03:21.298553 14778 tablet_bootstrap.cc:654] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.299049 14777 raft_consensus.cc:348] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.299360 14777 raft_consensus.cc:374] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.299474 14777 raft_consensus.cc:729] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b07a78f42ca4880bbe2f912fddce8f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.299682 14777 consensus_queue.cc:260] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.300559 14777 ts_tablet_manager.cc:1428] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 15:03:21.300690 14778 tablet_bootstrap.cc:492] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: No bootstrap required, opened a new log
I20260808 15:03:21.300470 14789 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.300809 14789 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:03:21.300899 14778 ts_tablet_manager.cc:1397] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 15:03:21.301578 14778 raft_consensus.cc:348] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.301757 14778 raft_consensus.cc:374] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.301877 14778 raft_consensus.cc:729] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ce5a7167af4b5b909db9eb814bbfd0, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.302080 14778 consensus_queue.cc:260] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.303555 14791 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.303952 14791 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:03:21.304366 14778 ts_tablet_manager.cc:1428] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:21.355597 14723 raft_consensus.cc:482] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:21.355773 14723 raft_consensus.cc:504] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.356302 14723 leader_election.cc:288] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337), 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859)
I20260808 15:03:21.356755 14158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f31ea4807d44aa5a1c8e2015378b5ac" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" is_pre_election: true
I20260808 15:03:21.356786 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f31ea4807d44aa5a1c8e2015378b5ac" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" is_pre_election: true
I20260808 15:03:21.357014 14158 raft_consensus.cc:2454] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 0.
I20260808 15:03:21.357014 14374 raft_consensus.cc:2454] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 0.
I20260808 15:03:21.357405 14475 leader_election.cc:302] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6e0d9698c64d05b404c6ea5ee468f5, 4b07a78f42ca4880bbe2f912fddce8f5; no voters:
I20260808 15:03:21.357681 14723 raft_consensus.cc:2781] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:03:21.357762 14723 raft_consensus.cc:482] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:03:21.357854 14723 raft_consensus.cc:3037] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.359714 14723 raft_consensus.cc:504] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.360227 14723 leader_election.cc:288] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 election: Requested vote from peers 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337), 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859)
I20260808 15:03:21.360526 14158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f31ea4807d44aa5a1c8e2015378b5ac" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ce5a7167af4b5b909db9eb814bbfd0"
I20260808 15:03:21.360666 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9f31ea4807d44aa5a1c8e2015378b5ac" candidate_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5"
I20260808 15:03:21.360769 14158 raft_consensus.cc:3037] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.360872 14374 raft_consensus.cc:3037] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.363504 14158 raft_consensus.cc:2454] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 1.
I20260808 15:03:21.363600 14374 raft_consensus.cc:2454] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0c6e0d9698c64d05b404c6ea5ee468f5 in term 1.
I20260808 15:03:21.363922 14473 leader_election.cc:302] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6e0d9698c64d05b404c6ea5ee468f5, 61ce5a7167af4b5b909db9eb814bbfd0; no voters:
I20260808 15:03:21.364159 14723 raft_consensus.cc:2781] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:21.364318 14723 raft_consensus.cc:686] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 LEADER]: Becoming Leader. State: Replica: 0c6e0d9698c64d05b404c6ea5ee468f5, State: Running, Role: LEADER
I20260808 15:03:21.364558 14723 consensus_queue.cc:237] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.366030 14757 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: TxnStatusTablet state changed. Reason: New leader 0c6e0d9698c64d05b404c6ea5ee468f5. Latest consensus state: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.366359 14757 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:03:21.366968 14793 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 15:03:21.367199 14793 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 15:03:21.367287 13919 catalog_manager.cc:5495] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } health_report { overall_health: HEALTHY } } }
I20260808 15:03:21.398813 14770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:21.392937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:45383 (local address 127.0.0.1:43604)
0808 15:03:21.393305 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:21.393328 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:21.393467 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:21.394040 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:21.394043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:21.394054 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:03:21.394203 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:21.394210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.395698 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.395701 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:21.396717 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:21.396730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.397684 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.397688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:21.397815 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.397871 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:03:21.398307 (+ 436us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:03:21.398324 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 15:03:21.398423 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 15:03:21.398547 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260808 15:03:21.398950 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:21.393070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43604 (local address 127.2.180.195:45383)
0808 15:03:21.393715 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:21.393722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:21.393747 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:03:21.393811 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:21.393816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:21.393895 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:21.394023 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:21.394372 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.395587 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.396859 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.397544 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.397682 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.398044 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:21.398175 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:21.398280 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:21.398472 (+ 192us) server_negotiation.cc:299] Negotiation successful
0808 15:03:21.398552 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":344,"threads_started":1}
I20260808 15:03:21.402755 14374 raft_consensus.cc:1261] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Refusing update from remote peer 0c6e0d9698c64d05b404c6ea5ee468f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:21.402850 14158 raft_consensus.cc:1261] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Refusing update from remote peer 0c6e0d9698c64d05b404c6ea5ee468f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:21.403301 14757 consensus_queue.cc:1035] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:21.403617 14723 consensus_queue.cc:1035] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:21.408515 14789 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: TxnStatusTablet state changed. Reason: New leader 0c6e0d9698c64d05b404c6ea5ee468f5. Latest consensus state: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.408787 14789 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:03:21.412261 14757 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.412518 14757 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:03:21.413373 14789 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.413594 14789 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 4b07a78f42ca4880bbe2f912fddce8f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:03:21.417131 14791 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: TxnStatusTablet state changed. Reason: New leader 0c6e0d9698c64d05b404c6ea5ee468f5. Latest consensus state: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.417419 14791 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:03:21.418795 14723 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.419081 14723 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 0c6e0d9698c64d05b404c6ea5ee468f5: This TxnStatusTablet replica's current role is: LEADER
I20260808 15:03:21.421403 14791 tablet_replica.cc:439] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } } }
I20260808 15:03:21.421640 14791 tablet_replica.cc:442] T 9f31ea4807d44aa5a1c8e2015378b5ac P 61ce5a7167af4b5b909db9eb814bbfd0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 15:03:21.436131 14796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:21.430838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:40331 (local address 127.2.180.195:45383)
0808 15:03:21.430975 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:21.430979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:21.431426 (+ 447us) server_negotiation.cc:407] Connection header received
0808 15:03:21.431608 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:21.431613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:21.431672 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:21.431744 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:21.432266 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.433009 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.434509 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.434912 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.435017 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.435422 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:21.435515 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:21.435586 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:21.435705 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 15:03:21.435792 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:03:21.436268 14814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:21.430776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:45383 (local address 127.2.180.195:40331)
0808 15:03:21.431332 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:21.431347 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:21.431477 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:21.431768 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:21.431772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:21.431783 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:03:21.432112 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:21.432121 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.433157 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.433163 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:21.434372 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:21.434385 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.435084 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.435093 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:21.435217 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.435270 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:03:21.435606 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:03:21.435616 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 15:03:21.435681 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 15:03:21.435839 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":1,"thread_start_us":317,"threads_started":1}
I20260808 15:03:21.445195 14374 raft_consensus.cc:1261] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Refusing update from remote peer 0c6e0d9698c64d05b404c6ea5ee468f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:21.445699 14158 raft_consensus.cc:1261] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Refusing update from remote peer 0c6e0d9698c64d05b404c6ea5ee468f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:21.445848 14723 consensus_queue.cc:1035] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:21.446276 14723 consensus_queue.cc:1035] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:21.511246 13916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:21.511456 13916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:21.514354 13916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: Sending DeleteTablet for 3 replicas of tablet 2372428266004959852ea451c7533057
I20260808 15:03:21.515017 14128 tablet_service.cc:1514] Processing DeleteTablet for tablet 2372428266004959852ea451c7533057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:21 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:34354
I20260808 15:03:21.515312 14337 tablet_service.cc:1514] Processing DeleteTablet for tablet 2372428266004959852ea451c7533057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:21 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:38578
I20260808 15:03:21.515491 14548 tablet_service.cc:1514] Processing DeleteTablet for tablet 2372428266004959852ea451c7533057 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:21 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:43586
I20260808 15:03:21.525342 14826 tablet_replica.cc:330] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: stopping tablet replica
I20260808 15:03:21.525517 14827 tablet_replica.cc:330] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: stopping tablet replica
I20260808 15:03:21.525732 14827 raft_consensus.cc:2229] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:21.525703 14826 raft_consensus.cc:2229] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:21.526022 14827 raft_consensus.cc:2258] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:21.526041 14826 raft_consensus.cc:2258] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:21.527141 14827 ts_tablet_manager.cc:1905] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:21.527158 14826 ts_tablet_manager.cc:1905] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:21.534049 14827 ts_tablet_manager.cc:1918] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:03:21.534047 14826 ts_tablet_manager.cc:1918] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:03:21.534176 14826 log.cc:1196] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/2372428266004959852ea451c7533057
I20260808 15:03:21.534261 14827 log.cc:1196] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/2372428266004959852ea451c7533057
I20260808 15:03:21.534726 14827 ts_tablet_manager.cc:1939] T 2372428266004959852ea451c7533057 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting consensus metadata
I20260808 15:03:21.534719 14826 ts_tablet_manager.cc:1939] T 2372428266004959852ea451c7533057 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting consensus metadata
I20260808 15:03:21.535776 13895 catalog_manager.cc:4841] TS 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337): tablet 2372428266004959852ea451c7533057 (table test-table [id=358d32fb304b4b208a96dc81a8ecb5b6]) successfully deleted
I20260808 15:03:21.535976 13899 catalog_manager.cc:4841] TS 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859): tablet 2372428266004959852ea451c7533057 (table test-table [id=358d32fb304b4b208a96dc81a8ecb5b6]) successfully deleted
I20260808 15:03:21.540974 14828 tablet_replica.cc:330] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: stopping tablet replica
I20260808 15:03:21.541170 14828 raft_consensus.cc:2229] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:03:21.541544 14828 raft_consensus.cc:2258] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:21.542413 14828 ts_tablet_manager.cc:1905] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:21.547573 14828 ts_tablet_manager.cc:1918] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:03:21.547693 14828 log.cc:1196] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/2372428266004959852ea451c7533057
I20260808 15:03:21.548094 14828 ts_tablet_manager.cc:1939] T 2372428266004959852ea451c7533057 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting consensus metadata
I20260808 15:03:21.548996 13899 catalog_manager.cc:4841] TS 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383): tablet 2372428266004959852ea451c7533057 (table test-table [id=358d32fb304b4b208a96dc81a8ecb5b6]) successfully deleted
Aug 08 15:03:21 dist-test-slave-jlzn krb5kdc[13853](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 15:03:21 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:21 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 15:03:21.590591 14769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:21.579534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35974 (local address 127.2.180.254:40235)
0808 15:03:21.579744 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:21.579751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:21.580188 (+ 437us) server_negotiation.cc:407] Connection header received
0808 15:03:21.580375 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:21.580381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:21.580480 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:21.580604 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:21.581823 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.582965 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.584221 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.584862 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.584991 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.590285 (+ 5294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35974: BlockingRecv error: recv got EOF from 127.0.0.1:35974 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 15:03:21 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM
I20260808 15:03:21.609488 14769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:21.593408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35978 (local address 127.2.180.254:40235)
0808 15:03:21.593573 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:21.593579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:21.596208 (+ 2629us) server_negotiation.cc:407] Connection header received
0808 15:03:21.596323 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:21.596328 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:21.596397 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:21.596482 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:21.597512 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.598614 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.599934 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.600580 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.600705 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.604520 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:21.604525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:21.604528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:21.604571 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:21.606974 (+ 2403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:21.607703 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:21.607707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:21.607709 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:21.607899 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:21.608275 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:21.608278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:21.608280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:21.608540 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:21.608754 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:21.609017 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 15:03:21.609210 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:03:21.612802 13916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:03:21.613519 13916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:03:21.620210 14128 tablet_service.cc:1467] Processing CreateTablet for tablet 05d075610bfe4d4c89cb5373de7eda44 (DEFAULT_TABLE table=test-table [id=461f211692c34a4a905abc443600cde7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:21.620333 14337 tablet_service.cc:1467] Processing CreateTablet for tablet 05d075610bfe4d4c89cb5373de7eda44 (DEFAULT_TABLE table=test-table [id=461f211692c34a4a905abc443600cde7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:21.620692 14128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05d075610bfe4d4c89cb5373de7eda44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.620707 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05d075610bfe4d4c89cb5373de7eda44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.627022 14777 tablet_bootstrap.cc:492] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: Bootstrap starting.
I20260808 15:03:21.627835 14548 tablet_service.cc:1467] Processing CreateTablet for tablet 05d075610bfe4d4c89cb5373de7eda44 (DEFAULT_TABLE table=test-table [id=461f211692c34a4a905abc443600cde7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:21.628428 14548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05d075610bfe4d4c89cb5373de7eda44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.628681 14777 tablet_bootstrap.cc:654] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.629705 14778 tablet_bootstrap.cc:492] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: Bootstrap starting.
I20260808 15:03:21.631283 14778 tablet_bootstrap.cc:654] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.633320 14777 tablet_bootstrap.cc:492] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: No bootstrap required, opened a new log
I20260808 15:03:21.633473 14777 ts_tablet_manager.cc:1397] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260808 15:03:21.634135 14771 tablet_bootstrap.cc:492] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: Bootstrap starting.
I20260808 15:03:21.634217 14777 raft_consensus.cc:348] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.634466 14777 raft_consensus.cc:374] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.634552 14777 raft_consensus.cc:729] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b07a78f42ca4880bbe2f912fddce8f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.634799 14777 consensus_queue.cc:260] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.635320 14777 ts_tablet_manager.cc:1428] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 15:03:21.635703 14771 tablet_bootstrap.cc:654] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.636186 14778 tablet_bootstrap.cc:492] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: No bootstrap required, opened a new log
I20260808 15:03:21.636370 14778 ts_tablet_manager.cc:1397] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260808 15:03:21.637140 14778 raft_consensus.cc:348] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.637472 14778 raft_consensus.cc:374] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.637601 14778 raft_consensus.cc:729] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ce5a7167af4b5b909db9eb814bbfd0, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.637879 14778 consensus_queue.cc:260] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.638767 14771 tablet_bootstrap.cc:492] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: No bootstrap required, opened a new log
I20260808 15:03:21.638942 14771 ts_tablet_manager.cc:1397] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 15:03:21.639338 14778 ts_tablet_manager.cc:1428] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 15:03:21.639847 14771 raft_consensus.cc:348] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.640162 14771 raft_consensus.cc:374] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.640287 14771 raft_consensus.cc:729] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6e0d9698c64d05b404c6ea5ee468f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.640592 14771 consensus_queue.cc:260] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.641981 14771 ts_tablet_manager.cc:1428] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 15:03:21.702328 14791 raft_consensus.cc:482] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:21.702601 14791 raft_consensus.cc:504] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.703444 14791 leader_election.cc:288] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859), 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383)
I20260808 15:03:21.703984 14583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05d075610bfe4d4c89cb5373de7eda44" candidate_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" is_pre_election: true
I20260808 15:03:21.703989 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05d075610bfe4d4c89cb5373de7eda44" candidate_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" is_pre_election: true
I20260808 15:03:21.704252 14374 raft_consensus.cc:2454] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61ce5a7167af4b5b909db9eb814bbfd0 in term 0.
I20260808 15:03:21.704285 14583 raft_consensus.cc:2454] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61ce5a7167af4b5b909db9eb814bbfd0 in term 0.
I20260808 15:03:21.704721 14059 leader_election.cc:302] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b07a78f42ca4880bbe2f912fddce8f5, 61ce5a7167af4b5b909db9eb814bbfd0; no voters:
I20260808 15:03:21.704972 14791 raft_consensus.cc:2781] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:03:21.705117 14791 raft_consensus.cc:482] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:03:21.705204 14791 raft_consensus.cc:3037] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.707280 14791 raft_consensus.cc:504] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.707836 14791 leader_election.cc:288] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [CANDIDATE]: Term 1 election: Requested vote from peers 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859), 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383)
I20260808 15:03:21.708269 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05d075610bfe4d4c89cb5373de7eda44" candidate_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b07a78f42ca4880bbe2f912fddce8f5"
I20260808 15:03:21.708396 14583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05d075610bfe4d4c89cb5373de7eda44" candidate_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5"
I20260808 15:03:21.708635 14583 raft_consensus.cc:3037] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.708647 14374 raft_consensus.cc:3037] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:21.710896 14374 raft_consensus.cc:2454] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61ce5a7167af4b5b909db9eb814bbfd0 in term 1.
I20260808 15:03:21.711262 14059 leader_election.cc:302] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b07a78f42ca4880bbe2f912fddce8f5, 61ce5a7167af4b5b909db9eb814bbfd0; no voters:
I20260808 15:03:21.711534 14791 raft_consensus.cc:2781] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:21.712021 14583 raft_consensus.cc:2454] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61ce5a7167af4b5b909db9eb814bbfd0 in term 1.
I20260808 15:03:21.712031 14791 raft_consensus.cc:686] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 LEADER]: Becoming Leader. State: Replica: 61ce5a7167af4b5b909db9eb814bbfd0, State: Running, Role: LEADER
I20260808 15:03:21.712601 14791 consensus_queue.cc:237] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.716169 13916 catalog_manager.cc:5495] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193). New cstate: current_term: 1 leader_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } health_report { overall_health: UNKNOWN } } }
I20260808 15:03:21.739919 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:21.733439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42578 (local address 127.2.180.193:34337)
0808 15:03:21.734025 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:21.734030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:21.734049 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:03:21.734133 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:21.734136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:21.734209 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:21.734337 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:21.734818 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.735548 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.737498 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:21.737964 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:21.738083 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:21.738610 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:21.739248 (+ 638us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:21.739354 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:21.739514 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 15:03:21.739583 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":358,"threads_started":1}
I20260808 15:03:21.743723 14374 raft_consensus.cc:1261] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Refusing update from remote peer 61ce5a7167af4b5b909db9eb814bbfd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:21.743827 14583 raft_consensus.cc:1261] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Refusing update from remote peer 61ce5a7167af4b5b909db9eb814bbfd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:21.744524 14857 consensus_queue.cc:1035] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:21.744892 14791 consensus_queue.cc:1035] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:21.773017 13919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:21.773274 13919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:21.777757 13919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: Sending DeleteTablet for 3 replicas of tablet 05d075610bfe4d4c89cb5373de7eda44
I20260808 15:03:21.779066 14337 tablet_service.cc:1514] Processing DeleteTablet for tablet 05d075610bfe4d4c89cb5373de7eda44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:21 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:38578
I20260808 15:03:21.779361 14548 tablet_service.cc:1514] Processing DeleteTablet for tablet 05d075610bfe4d4c89cb5373de7eda44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:21 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:43586
I20260808 15:03:21.779640 14827 tablet_replica.cc:330] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: stopping tablet replica
I20260808 15:03:21.779861 14827 raft_consensus.cc:2229] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:21.780071 14827 raft_consensus.cc:2258] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:21.780340 14828 tablet_replica.cc:330] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: stopping tablet replica
I20260808 15:03:21.780530 14828 raft_consensus.cc:2229] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:21.780748 14828 raft_consensus.cc:2258] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:21.781935 14128 tablet_service.cc:1514] Processing DeleteTablet for tablet 05d075610bfe4d4c89cb5373de7eda44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:21 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:34354
I20260808 15:03:21.782204 14826 tablet_replica.cc:330] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: stopping tablet replica
I20260808 15:03:21.782424 14826 raft_consensus.cc:2229] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:03:21.782864 14826 raft_consensus.cc:2258] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:21.783885 14826 ts_tablet_manager.cc:1905] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:21.784154 13919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 15:03:21.784703 14828 ts_tablet_manager.cc:1905] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 15:03:21.784778 13919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:03:21.790753 14826 ts_tablet_manager.cc:1918] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:21.790947 14826 log.cc:1196] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/05d075610bfe4d4c89cb5373de7eda44
I20260808 15:03:21.791389 14826 ts_tablet_manager.cc:1939] T 05d075610bfe4d4c89cb5373de7eda44 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting consensus metadata
I20260808 15:03:21.792543 14827 ts_tablet_manager.cc:1905] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:21.793007 13895 catalog_manager.cc:4841] TS 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337): tablet 05d075610bfe4d4c89cb5373de7eda44 (table test-table [id=461f211692c34a4a905abc443600cde7]) successfully deleted
I20260808 15:03:21.797866 14828 ts_tablet_manager.cc:1918] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:21.798084 14828 log.cc:1196] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/05d075610bfe4d4c89cb5373de7eda44
I20260808 15:03:21.798662 14828 ts_tablet_manager.cc:1939] T 05d075610bfe4d4c89cb5373de7eda44 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting consensus metadata
I20260808 15:03:21.800871 14827 ts_tablet_manager.cc:1918] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:21.800980 13899 catalog_manager.cc:4841] TS 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383): tablet 05d075610bfe4d4c89cb5373de7eda44 (table test-table [id=461f211692c34a4a905abc443600cde7]) successfully deleted
I20260808 15:03:21.801071 14827 log.cc:1196] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/05d075610bfe4d4c89cb5373de7eda44
I20260808 15:03:21.801483 14827 ts_tablet_manager.cc:1939] T 05d075610bfe4d4c89cb5373de7eda44 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting consensus metadata
I20260808 15:03:21.802567 13899 catalog_manager.cc:4841] TS 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859): tablet 05d075610bfe4d4c89cb5373de7eda44 (table test-table [id=461f211692c34a4a905abc443600cde7]) successfully deleted
I20260808 15:03:21.804250 14128 tablet_service.cc:1467] Processing CreateTablet for tablet 44772449006d4dd5bd2503dba326ebd7 (DEFAULT_TABLE table=test-table [id=463d2de3a3164c229c623f6002da30a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:21.804608 14548 tablet_service.cc:1467] Processing CreateTablet for tablet 44772449006d4dd5bd2503dba326ebd7 (DEFAULT_TABLE table=test-table [id=463d2de3a3164c229c623f6002da30a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:21.804714 14128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44772449006d4dd5bd2503dba326ebd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.804963 14548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44772449006d4dd5bd2503dba326ebd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.808910 14337 tablet_service.cc:1467] Processing CreateTablet for tablet 44772449006d4dd5bd2503dba326ebd7 (DEFAULT_TABLE table=test-table [id=463d2de3a3164c229c623f6002da30a5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:21.809258 14337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44772449006d4dd5bd2503dba326ebd7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:21.811434 14778 tablet_bootstrap.cc:492] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: Bootstrap starting.
I20260808 15:03:21.813211 14778 tablet_bootstrap.cc:654] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.814476 14777 tablet_bootstrap.cc:492] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: Bootstrap starting.
I20260808 15:03:21.815981 14777 tablet_bootstrap.cc:654] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.817652 14771 tablet_bootstrap.cc:492] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: Bootstrap starting.
I20260808 15:03:21.819293 14771 tablet_bootstrap.cc:654] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:21.826176 14771 tablet_bootstrap.cc:492] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: No bootstrap required, opened a new log
I20260808 15:03:21.826330 14771 ts_tablet_manager.cc:1397] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.001s
I20260808 15:03:21.826885 14778 tablet_bootstrap.cc:492] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: No bootstrap required, opened a new log
I20260808 15:03:21.826879 14777 tablet_bootstrap.cc:492] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: No bootstrap required, opened a new log
I20260808 15:03:21.827005 14777 ts_tablet_manager.cc:1397] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.007s
I20260808 15:03:21.827026 14778 ts_tablet_manager.cc:1397] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.009s
I20260808 15:03:21.827103 14771 raft_consensus.cc:348] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.827330 14771 raft_consensus.cc:374] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.827423 14771 raft_consensus.cc:729] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c6e0d9698c64d05b404c6ea5ee468f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.827719 14777 raft_consensus.cc:348] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.827745 14778 raft_consensus.cc:348] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.828043 14777 raft_consensus.cc:374] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.828061 14778 raft_consensus.cc:374] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:21.828154 14778 raft_consensus.cc:729] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61ce5a7167af4b5b909db9eb814bbfd0, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.828173 14777 raft_consensus.cc:729] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b07a78f42ca4880bbe2f912fddce8f5, State: Initialized, Role: FOLLOWER
I20260808 15:03:21.828411 14778 consensus_queue.cc:260] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.828505 14777 consensus_queue.cc:260] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.828999 14778 ts_tablet_manager.cc:1428] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260808 15:03:21.827862 14771 consensus_queue.cc:260] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:21.829748 14771 ts_tablet_manager.cc:1428] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:03:21.830423 14777 ts_tablet_manager.cc:1428] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 15:03:22.017023 14789 raft_consensus.cc:482] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:22.017233 14789 raft_consensus.cc:504] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:22.018754 14789 leader_election.cc:288] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337), 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383)
I20260808 15:03:22.040060 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.019605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:32911 (local address 127.2.180.193:34337)
0808 15:03:22.019838 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:22.019845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:22.030823 (+ 10978us) server_negotiation.cc:407] Connection header received
0808 15:03:22.031751 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:22.031758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:22.031858 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:22.031928 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:22.033285 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.034496 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.037205 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.038860 (+ 1655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.039028 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.039103 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:22.039646 (+ 543us) server_negotiation.cc:299] Negotiation successful
0808 15:03:22.039780 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 15:03:22.041004 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.018895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34337 (local address 127.2.180.194:32911)
0808 15:03:22.030696 (+ 11801us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:22.030718 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:22.031421 (+ 703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:22.032180 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:22.032185 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:22.032217 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:22.032536 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:22.032547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.034716 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.034721 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:22.037023 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:22.037037 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.039237 (+ 2200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.039244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:22.039446 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.039481 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 15:03:22.040417 (+ 936us) client_negotiation.cc:240] Negotiation successful
0808 15:03:22.040551 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11668,"mutex_wait_us":182,"thread_start_us":11533,"threads_started":1}
I20260808 15:03:22.041993 14157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44772449006d4dd5bd2503dba326ebd7" candidate_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" is_pre_election: true
I20260808 15:03:22.042420 14157 raft_consensus.cc:2454] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b07a78f42ca4880bbe2f912fddce8f5 in term 0.
I20260808 15:03:22.043133 14267 leader_election.cc:302] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b07a78f42ca4880bbe2f912fddce8f5, 61ce5a7167af4b5b909db9eb814bbfd0; no voters:
I20260808 15:03:22.043419 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.019334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:37929 (local address 127.2.180.195:45383)
0808 15:03:22.034918 (+ 15584us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:22.034925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:22.035664 (+ 739us) server_negotiation.cc:407] Connection header received
0808 15:03:22.035748 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:22.035753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:22.035843 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:22.035927 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:22.036746 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.037723 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.040051 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.041967 (+ 1916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.042142 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.042304 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:22.042948 (+ 644us) server_negotiation.cc:299] Negotiation successful
0808 15:03:22.043092 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15461,"thread_start_us":15584,"threads_started":1}
I20260808 15:03:22.043542 14789 raft_consensus.cc:2781] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:03:22.043648 14789 raft_consensus.cc:482] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:03:22.043776 14789 raft_consensus.cc:3037] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:22.044150 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.019224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:45383 (local address 127.2.180.194:37929)
0808 15:03:22.035405 (+ 16181us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:22.035424 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:22.035546 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:22.036111 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:22.036116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:22.036141 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:22.036556 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:22.036570 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.037883 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.037888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:22.039890 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:22.039904 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.042482 (+ 2578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.042488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:22.042653 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.042685 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:03:22.042771 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 15:03:22.043687 (+ 916us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16054,"mutex_wait_us":4,"thread_start_us":17021,"threads_started":1}
I20260808 15:03:22.044719 14583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44772449006d4dd5bd2503dba326ebd7" candidate_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" is_pre_election: true
I20260808 15:03:22.044950 14583 raft_consensus.cc:2454] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b07a78f42ca4880bbe2f912fddce8f5 in term 0.
I20260808 15:03:22.046945 14789 raft_consensus.cc:504] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:22.047586 14789 leader_election.cc:288] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [CANDIDATE]: Term 1 election: Requested vote from peers 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337), 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383)
I20260808 15:03:22.048163 14583 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44772449006d4dd5bd2503dba326ebd7" candidate_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5"
I20260808 15:03:22.048381 14583 raft_consensus.cc:3037] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:22.048497 14157 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44772449006d4dd5bd2503dba326ebd7" candidate_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61ce5a7167af4b5b909db9eb814bbfd0"
I20260808 15:03:22.048708 14157 raft_consensus.cc:3037] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:22.051199 14583 raft_consensus.cc:2454] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b07a78f42ca4880bbe2f912fddce8f5 in term 1.
I20260808 15:03:22.051733 14269 leader_election.cc:302] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0c6e0d9698c64d05b404c6ea5ee468f5, 4b07a78f42ca4880bbe2f912fddce8f5; no voters:
I20260808 15:03:22.051977 14157 raft_consensus.cc:2454] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b07a78f42ca4880bbe2f912fddce8f5 in term 1.
I20260808 15:03:22.052537 14789 raft_consensus.cc:2781] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:22.053785 14789 raft_consensus.cc:686] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 LEADER]: Becoming Leader. State: Replica: 4b07a78f42ca4880bbe2f912fddce8f5, State: Running, Role: LEADER
I20260808 15:03:22.054430 14789 consensus_queue.cc:237] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } }
I20260808 15:03:22.058622 13915 catalog_manager.cc:5495] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194). New cstate: current_term: 1 leader_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b07a78f42ca4880bbe2f912fddce8f5" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 } health_report { overall_health: UNKNOWN } } }
I20260808 15:03:22.129876 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.123446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37742 (local address 127.2.180.194:45859)
0808 15:03:22.124027 (+ 581us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:22.124031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:22.124048 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:03:22.124124 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:22.124127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:22.124205 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:22.124334 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:22.124837 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.125613 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.127575 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.128044 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.128136 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.128682 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:22.129252 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:22.129363 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:22.129496 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 15:03:22.129559 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":348,"threads_started":1}
I20260808 15:03:22.145500 14888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.139689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:45383 (local address 127.2.180.194:53993)
0808 15:03:22.140265 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:22.140279 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:22.140406 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:22.140700 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:22.140704 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:22.140717 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:03:22.141006 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:22.141014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.142405 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.142408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:22.143555 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:22.143569 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.144506 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.144511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:22.144626 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.144677 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:03:22.145060 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:03:22.145066 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:03:22.145119 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 15:03:22.145224 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":277,"threads_started":1}
I20260808 15:03:22.145493 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.139741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:53993 (local address 127.2.180.195:45383)
0808 15:03:22.139936 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:22.139942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:22.140360 (+ 418us) server_negotiation.cc:407] Connection header received
0808 15:03:22.140526 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:22.140531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:22.140595 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:22.140672 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:22.141197 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.142280 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.143698 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.144333 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.144464 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.144846 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:22.144971 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:22.145079 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:22.145144 (+ 65us) server_negotiation.cc:299] Negotiation successful
0808 15:03:22.145225 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:03:22.154228 14583 raft_consensus.cc:1261] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Refusing update from remote peer 4b07a78f42ca4880bbe2f912fddce8f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:22.154417 14157 raft_consensus.cc:1261] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Refusing update from remote peer 4b07a78f42ca4880bbe2f912fddce8f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:22.154908 14880 consensus_queue.cc:1035] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c6e0d9698c64d05b404c6ea5ee468f5" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:22.155265 14789 consensus_queue.cc:1035] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "61ce5a7167af4b5b909db9eb814bbfd0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34337 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:22.185559 14897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.178570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:45859 (local address 127.2.180.195:48621)
0808 15:03:22.180309 (+ 1739us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:22.180328 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:22.180462 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:22.180790 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:22.180795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:22.180806 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 15:03:22.181054 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:22.181063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.182403 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.182408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:22.183610 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:22.183624 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.184586 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.184591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:22.184733 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.184782 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 15:03:22.185149 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 15:03:22.185156 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 15:03:22.185207 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 15:03:22.185302 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1606,"thread_start_us":1464,"threads_started":1}
I20260808 15:03:22.185571 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.178783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:48621 (local address 127.2.180.194:45859)
0808 15:03:22.178949 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:22.178955 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:22.180402 (+ 1447us) server_negotiation.cc:407] Connection header received
0808 15:03:22.180600 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:22.180606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:22.180688 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:22.180777 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:22.181189 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.182265 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.183766 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.184444 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.184574 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.184921 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:22.185049 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:22.185147 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:22.185231 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 15:03:22.185310 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:03:22.218884 13915 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:22.219172 13915 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:22.222512 13915 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e5f1219241d40549cb74ac5491f9a60: Sending DeleteTablet for 3 replicas of tablet 44772449006d4dd5bd2503dba326ebd7
I20260808 15:03:22.223448 14128 tablet_service.cc:1514] Processing DeleteTablet for tablet 44772449006d4dd5bd2503dba326ebd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:22 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:34354
I20260808 15:03:22.223572 14548 tablet_service.cc:1514] Processing DeleteTablet for tablet 44772449006d4dd5bd2503dba326ebd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:22 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:43586
I20260808 15:03:22.223882 14828 tablet_replica.cc:330] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: stopping tablet replica
I20260808 15:03:22.223987 14826 tablet_replica.cc:330] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: stopping tablet replica
I20260808 15:03:22.224056 14828 raft_consensus.cc:2229] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:22.224180 14826 raft_consensus.cc:2229] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:22.224232 14828 raft_consensus.cc:2258] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:22.224378 14826 raft_consensus.cc:2258] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:22.224956 14337 tablet_service.cc:1514] Processing DeleteTablet for tablet 44772449006d4dd5bd2503dba326ebd7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:22 UTC) from {username='oryx', principal='oryx/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:38578
I20260808 15:03:22.225461 14828 ts_tablet_manager.cc:1905] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:22.226374 14827 tablet_replica.cc:330] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: stopping tablet replica
I20260808 15:03:22.226550 14827 raft_consensus.cc:2229] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:03:22.227026 14827 raft_consensus.cc:2258] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:22.227592 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 14030
I20260808 15:03:22.229498 14827 ts_tablet_manager.cc:1905] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:22.229553 14826 ts_tablet_manager.cc:1905] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:22.236402 14828 ts_tablet_manager.cc:1918] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:03:22.240973 14828 log.cc:1196] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/44772449006d4dd5bd2503dba326ebd7
I20260808 15:03:22.241554 14828 ts_tablet_manager.cc:1939] T 44772449006d4dd5bd2503dba326ebd7 P 0c6e0d9698c64d05b404c6ea5ee468f5: Deleting consensus metadata
I20260808 15:03:22.240839 14827 ts_tablet_manager.cc:1918] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:03:22.241683 14826 ts_tablet_manager.cc:1918] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 15:03:22.241835 14826 log.cc:1196] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/44772449006d4dd5bd2503dba326ebd7
I20260808 15:03:22.242115 14827 log.cc:1196] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/44772449006d4dd5bd2503dba326ebd7
I20260808 15:03:22.242265 14826 ts_tablet_manager.cc:1939] T 44772449006d4dd5bd2503dba326ebd7 P 61ce5a7167af4b5b909db9eb814bbfd0: Deleting consensus metadata
I20260808 15:03:22.243399 13899 catalog_manager.cc:4841] TS 0c6e0d9698c64d05b404c6ea5ee468f5 (127.2.180.195:45383): tablet 44772449006d4dd5bd2503dba326ebd7 (table test-table [id=463d2de3a3164c229c623f6002da30a5]) successfully deleted
I20260808 15:03:22.244051 14827 ts_tablet_manager.cc:1939] T 44772449006d4dd5bd2503dba326ebd7 P 4b07a78f42ca4880bbe2f912fddce8f5: Deleting consensus metadata
I20260808 15:03:22.245252 13899 catalog_manager.cc:4841] TS 4b07a78f42ca4880bbe2f912fddce8f5 (127.2.180.194:45859): tablet 44772449006d4dd5bd2503dba326ebd7 (table test-table [id=463d2de3a3164c229c623f6002da30a5]) successfully deleted
I20260808 15:03:22.245513 13895 catalog_manager.cc:4841] TS 61ce5a7167af4b5b909db9eb814bbfd0 (127.2.180.193:34337): tablet 44772449006d4dd5bd2503dba326ebd7 (table test-table [id=463d2de3a3164c229c623f6002da30a5]) successfully deleted
Aug 08 15:03:22 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.193@KRBTEST.COM
I20260808 15:03:22.259723 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.228754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42584 (local address 127.2.180.193:34337)
0808 15:03:22.228910 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:22.228917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:22.246254 (+ 17337us) server_negotiation.cc:407] Connection header received
0808 15:03:22.246427 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:22.246431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:22.246500 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:22.246578 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:22.247728 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.248509 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.249722 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.250128 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.250236 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.254324 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:22.254335 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:22.254350 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:22.254380 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:22.257190 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:22.257808 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:22.257816 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:22.257818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:22.257947 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:22.258297 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:22.258301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:22.258303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:22.258633 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:22.258902 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:22.259218 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 15:03:22.259415 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:03:22.260406 14225 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:22.405375 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 14030
I20260808 15:03:22.435680 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 14239
Aug 08 15:03:22 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.194@KRBTEST.COM
I20260808 15:03:22.451117 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.436440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.2.180.194:45859)
0808 15:03:22.436636 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:22.436642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:22.436840 (+ 198us) server_negotiation.cc:407] Connection header received
0808 15:03:22.437022 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:22.437026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:22.437118 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:22.437226 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:22.438510 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.439376 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.440749 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.441199 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.441318 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.445506 (+ 4188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:22.445516 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:22.445533 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:22.445567 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:22.448473 (+ 2906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:22.449103 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:22.449110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:22.449113 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:22.449271 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:22.449618 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:22.449622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:22.449625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:22.449964 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:22.450313 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:22.450517 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 15:03:22.450731 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:03:22.451745 14434 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:22.573812 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 14239
I20260808 15:03:22.594233 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 14447
Aug 08 15:03:22 dist-test-slave-jlzn krb5kdc[13853](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.2.180.195@KRBTEST.COM
I20260808 15:03:22.608793 14879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:22.595068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36832 (local address 127.2.180.195:45383)
0808 15:03:22.595245 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:22.595251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:22.595270 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:03:22.595363 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:22.595367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:22.595431 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:22.595516 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:22.596830 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.597545 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.598808 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:22.599224 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:22.599330 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:22.603614 (+ 4284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:22.603629 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:22.603645 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:22.603685 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:22.606197 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:22.606804 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:22.606812 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:22.606814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:22.606954 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:22.607304 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:22.607309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:22.607312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:22.607674 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:22.607995 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:22.608253 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 15:03:22.608461 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:03:22.609463 14660 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:22.751854 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 14447
I20260808 15:03:22.778878 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 13870
I20260808 15:03:22.780097 13988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:22.894261 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 13870
2026-08-08T15:03:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (7230 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:03:22 dist-test-slave-jlzn krb5kdc[14911](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:03:22 dist-test-slave-jlzn krb5kdc[14911](info): set up 2 sockets
Aug 08 15:03:22 dist-test-slave-jlzn krb5kdc[14911](info): commencing operation
krb5kdc: starting...
W20260808 15:03:24.067796 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:03:24 dist-test-slave-jlzn krb5kdc[14911](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:03:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:03:24.269274 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35825
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:35825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:24.581559 14933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:24.581840 14933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:24.581979 14933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:24.588877 14933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:24.588984 14933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:24.589066 14933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:24.589143 14933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:24.589208 14933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:24.598640 14933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:35825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:24.599818 14933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:24.601027 14933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:24.611934 14943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:24.611995 14945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:24.612131 14942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:24.612973 14933 server_base.cc:1034] running on GCE node
I20260808 15:03:24.613649 14933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:24.614883 14933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:24.616082 14933 hybrid_clock.cc:648] HybridClock initialized: now 1786201404616024 us; error 41 us; skew 500 ppm
I20260808 15:03:24.616393 14933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T15:03:24Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1793 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:03:24 dist-test-slave-jlzn krb5kdc[14951](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:03:24 dist-test-slave-jlzn krb5kdc[14951](info): set up 2 sockets
Aug 08 15:03:24 dist-test-slave-jlzn krb5kdc[14951](info): commencing operation
krb5kdc: starting...
W20260808 15:03:25.870133 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:03:26 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:03:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:03:26.083103 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46055
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:46055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:26.393720 14967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:26.393975 14967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:26.394059 14967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:26.400985 14967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:26.401078 14967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:26.401129 14967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:26.401188 14967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:26.401232 14967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:26.410352 14967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:46055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:26.411522 14967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:26.412744 14967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:26.423556 14973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:26.423527 14975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:26.423733 14978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:26.424562 14967 server_base.cc:1034] running on GCE node
I20260808 15:03:26.425272 14967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:26.426522 14967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:26.427722 14967 hybrid_clock.cc:648] HybridClock initialized: now 1786201406427680 us; error 32 us; skew 500 ppm
I20260808 15:03:26.428042 14967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:26 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:26.430902 14967 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:03:26.432322 14967 webserver.cc:460] Webserver started at http://127.2.180.254:35871/ using document root <none> and password file <none>
I20260808 15:03:26.432935 14967 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:26.433055 14967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:26.433379 14967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:26.436244 14967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9"
format_stamp: "Formatted at 2026-08-08 15:03:26 on dist-test-slave-jlzn"
server_key: "f85f7112e37ec204673f4db22b115d0c"
server_key_iv: "a54d9ecaa184a7dcadb22b566df8c70e"
server_key_version: "encryptionkey@0"
I20260808 15:03:26.436980 14967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9"
format_stamp: "Formatted at 2026-08-08 15:03:26 on dist-test-slave-jlzn"
server_key: "f85f7112e37ec204673f4db22b115d0c"
server_key_iv: "a54d9ecaa184a7dcadb22b566df8c70e"
server_key_version: "encryptionkey@0"
I20260808 15:03:26.441764 14967 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:03:26.445017 14989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:26.446233 14967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:26.446395 14967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9"
format_stamp: "Formatted at 2026-08-08 15:03:26 on dist-test-slave-jlzn"
server_key: "f85f7112e37ec204673f4db22b115d0c"
server_key_iv: "a54d9ecaa184a7dcadb22b566df8c70e"
server_key_version: "encryptionkey@0"
I20260808 15:03:26.446570 14967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:26.462124 14967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:26.465770 14967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:26.466019 14967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:26.482890 14967 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:46055
I20260808 15:03:26.482913 15077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:46055 every 8 connection(s)
I20260808 15:03:26.484383 14967 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:26.488237 15080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:26.490931 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 14967
I20260808 15:03:26.491112 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:26.491770 2771 external_mini_cluster.cc:1428] Setting key d2755b38c954e82e4d156798013b7726
I20260808 15:03:26.500017 15080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Bootstrap starting.
I20260808 15:03:26.503613 15080 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:26.504884 15080 log.cc:824] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:03:26 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:26.507972 15080 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: No bootstrap required, opened a new log
I20260808 15:03:26.515031 15085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:26.493646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.2.180.254:46055)
0808 15:03:26.494355 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:26.494366 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:26.494406 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:03:26.496073 (+ 1667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:26.496095 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:26.496474 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:26.496822 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:26.498563 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:26.499883 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:26.501574 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:26.502228 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:26.502345 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:26.506958 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:26.506968 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:26.506984 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:26.507031 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:26.510838 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:26.511461 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:26.511469 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:26.511477 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:26.511714 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:26.512067 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:26.512072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:26.512076 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:26.512749 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:26.513059 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:26.513330 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 15:03:26.513755 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":304,"threads_started":1}
I20260808 15:03:26.517944 15080 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } }
I20260808 15:03:26.518275 15080 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:26.518345 15080 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce5a8555a7584ae8ab3a3a2dde2010e9, State: Initialized, Role: FOLLOWER
I20260808 15:03:26.518828 15080 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } }
I20260808 15:03:26.519032 15080 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:26.519130 15080 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:26.519260 15080 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:26.521405 15080 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } }
I20260808 15:03:26.521821 15080 leader_election.cc:302] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce5a8555a7584ae8ab3a3a2dde2010e9; no voters:
I20260808 15:03:26.522337 15080 leader_election.cc:288] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:26.522459 15088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:26.523106 15088 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 1 LEADER]: Becoming Leader. State: Replica: ce5a8555a7584ae8ab3a3a2dde2010e9, State: Running, Role: LEADER
I20260808 15:03:26.523591 15088 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } }
I20260808 15:03:26.524158 15080 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:26.526232 15089 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } } }
I20260808 15:03:26.526598 15089 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:26.527046 15090 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce5a8555a7584ae8ab3a3a2dde2010e9. Latest consensus state: current_term: 1 leader_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } } }
I20260808 15:03:26.527320 15090 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:26.528390 15099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:26.533382 15099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:26.541714 15099 catalog_manager.cc:1347] Generated new cluster ID: ab85f4890dba4f29918298ee3ffb7fcb
I20260808 15:03:26.541814 15099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:26.555608 15099 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:26.557369 15099 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:26.565164 15099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Generated new TSK 0
I20260808 15:03:26.565848 15099 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:03:26.639029 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46055
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:26.952888 15116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:26.953131 15116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:26.953227 15116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:26.960069 15116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:26.960162 15116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:26.960270 15116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:26.969331 15116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:26.970366 15116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:26.971603 15116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:26.983425 15123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:26.983595 15126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:26.983618 15122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:26.984560 15116 server_base.cc:1034] running on GCE node
I20260808 15:03:26.985111 15116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:26.985914 15116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:26.987107 15116 hybrid_clock.cc:648] HybridClock initialized: now 1786201406987070 us; error 24 us; skew 500 ppm
I20260808 15:03:26.987380 15116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:26 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:26.990280 15116 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:03:26.991611 15116 webserver.cc:460] Webserver started at http://127.2.180.193:36573/ using document root <none> and password file <none>
I20260808 15:03:26.992259 15116 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:26.992365 15116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:26.992657 15116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:26.995283 15116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "66e83c71940b4b9681b9ca1d960d2709"
format_stamp: "Formatted at 2026-08-08 15:03:26 on dist-test-slave-jlzn"
server_key: "d56a563287e7856dcb0a3f20083dfc4b"
server_key_iv: "f6e7b55561159ceab86588150a88a521"
server_key_version: "encryptionkey@0"
I20260808 15:03:26.996076 15116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "66e83c71940b4b9681b9ca1d960d2709"
format_stamp: "Formatted at 2026-08-08 15:03:26 on dist-test-slave-jlzn"
server_key: "d56a563287e7856dcb0a3f20083dfc4b"
server_key_iv: "f6e7b55561159ceab86588150a88a521"
server_key_version: "encryptionkey@0"
I20260808 15:03:27.000838 15116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:03:27.003988 15134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:27.005141 15116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:03:27.005280 15116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "66e83c71940b4b9681b9ca1d960d2709"
format_stamp: "Formatted at 2026-08-08 15:03:26 on dist-test-slave-jlzn"
server_key: "d56a563287e7856dcb0a3f20083dfc4b"
server_key_iv: "f6e7b55561159ceab86588150a88a521"
server_key_version: "encryptionkey@0"
I20260808 15:03:27.005455 15116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:27.032438 15116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:27.036266 15116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:27.036501 15116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:27.037365 15116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:27.038580 15116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:27.038653 15116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:27.038761 15116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:27.038810 15116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:27.068446 15116 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:33981
I20260808 15:03:27.068509 15311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:33981 every 8 connection(s)
I20260808 15:03:27.069563 15116 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:27.077934 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 15116
I20260808 15:03:27.078115 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:03:27.078755 2771 external_mini_cluster.cc:1428] Setting key ff407c18adcdaf47e120150a2217d661
Aug 08 15:03:27 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201406, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:27.088376 15315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:27.071837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:60447 (local address 127.2.180.254:46055)
0808 15:03:27.072297 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:27.072303 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:27.073198 (+ 895us) server_negotiation.cc:407] Connection header received
0808 15:03:27.074875 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:27.074880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:27.074946 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:27.075052 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:27.076432 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:27.077159 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:27.078682 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:27.079257 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:27.079397 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:27.082744 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:27.082751 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:27.082756 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:27.082793 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:27.085612 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:27.086264 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:27.086269 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:27.086272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:27.086419 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:27.086769 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:27.086772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:27.086775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:27.087079 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:27.087232 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:27.087827 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 15:03:27.088040 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":199,"threads_started":1}
I20260808 15:03:27.089010 15316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:27.072378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:60447)
0808 15:03:27.073033 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:27.073090 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:27.074645 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:27.075242 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:27.075250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:27.075722 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:27.076298 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:27.076316 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:27.077274 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:27.077277 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:27.078538 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:27.078555 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:27.079473 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:27.079481 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:27.079730 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:27.080352 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:27.080380 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:27.082531 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:27.085735 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:27.085741 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:27.085751 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:27.086143 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:27.086503 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:27.086506 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:27.086508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:27.086669 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:27.087178 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:27.087183 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:27.087455 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 15:03:27.087660 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 15:03:27.088038 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":257,"threads_started":1}
I20260808 15:03:27.090363 15312 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46055
I20260808 15:03:27.090629 15312 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:27.091279 15312 heartbeater.cc:511] Master 127.2.180.254:46055 requested a full tablet report, sending...
I20260808 15:03:27.093317 15018 ts_manager.cc:194] Registered new tserver with Master: 66e83c71940b4b9681b9ca1d960d2709 (127.2.180.193:33981)
I20260808 15:03:27.094712 15018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:60447
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:03:27.128984 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46055
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:27.437328 15324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:27.437567 15324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:27.437687 15324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:27.444491 15324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:27.444587 15324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:27.444746 15324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:27.453814 15324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:27.454897 15324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:27.456094 15324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:27.467968 15330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:27.468052 15333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:27.468052 15331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:27.468987 15324 server_base.cc:1034] running on GCE node
I20260808 15:03:27.469547 15324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:27.470414 15324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:27.471614 15324 hybrid_clock.cc:648] HybridClock initialized: now 1786201407471559 us; error 47 us; skew 500 ppm
I20260808 15:03:27.471918 15324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:27 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:27.474779 15324 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:03:27.476105 15324 webserver.cc:460] Webserver started at http://127.2.180.194:38513/ using document root <none> and password file <none>
I20260808 15:03:27.476712 15324 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:27.476823 15324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:27.477113 15324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:27.479916 15324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "8665747ff23c477ba2156e8e4d6c23a0"
format_stamp: "Formatted at 2026-08-08 15:03:27 on dist-test-slave-jlzn"
server_key: "e225c83d77741da4c947d862ad5d78b3"
server_key_iv: "b81f7c726f1c5240a8b80cb4bdf9d80e"
server_key_version: "encryptionkey@0"
I20260808 15:03:27.480664 15324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "8665747ff23c477ba2156e8e4d6c23a0"
format_stamp: "Formatted at 2026-08-08 15:03:27 on dist-test-slave-jlzn"
server_key: "e225c83d77741da4c947d862ad5d78b3"
server_key_iv: "b81f7c726f1c5240a8b80cb4bdf9d80e"
server_key_version: "encryptionkey@0"
I20260808 15:03:27.485549 15324 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 15:03:27.488876 15341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:27.490094 15324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:03:27.490238 15324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "8665747ff23c477ba2156e8e4d6c23a0"
format_stamp: "Formatted at 2026-08-08 15:03:27 on dist-test-slave-jlzn"
server_key: "e225c83d77741da4c947d862ad5d78b3"
server_key_iv: "b81f7c726f1c5240a8b80cb4bdf9d80e"
server_key_version: "encryptionkey@0"
I20260808 15:03:27.490406 15324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:27.502988 15324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:27.506619 15324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:27.506873 15324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:27.507699 15324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:27.508991 15324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:27.509068 15324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:27.509182 15324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:27.509244 15324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:27.538050 15324 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:40587
I20260808 15:03:27.538087 15510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:40587 every 8 connection(s)
I20260808 15:03:27.539240 15324 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:27.546443 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 15324
I20260808 15:03:27.546586 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:03:27.547003 2771 external_mini_cluster.cc:1428] Setting key c80fe2175d5e378ee36df24887775299
Aug 08 15:03:27 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:27.558310 15315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:27.541620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:50467 (local address 127.2.180.254:46055)
0808 15:03:27.541793 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:27.541797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:27.542876 (+ 1079us) server_negotiation.cc:407] Connection header received
0808 15:03:27.544622 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:27.544627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:27.544706 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:27.544791 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:27.546221 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:27.547003 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:27.548854 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:27.549318 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:27.549411 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:27.553083 (+ 3672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:27.553087 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:27.553090 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:27.553120 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:27.555464 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:27.556171 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:27.556175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:27.556177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:27.556306 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:27.556711 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:27.556714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:27.556716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:27.556967 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:27.557112 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:27.557771 (+ 659us) server_negotiation.cc:299] Negotiation successful
0808 15:03:27.557991 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:03:27.559122 15513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:27.542035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:50467)
0808 15:03:27.542731 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:27.542766 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:27.544366 (+ 1600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:27.545013 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:27.545021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:27.545507 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:27.546041 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:27.546058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:27.547163 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:27.547169 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:27.548712 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:27.548727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:27.549540 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:27.549548 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:27.549855 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:27.550500 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:27.550528 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:27.552882 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:27.555597 (+ 2715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:27.555606 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:27.555619 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:27.556055 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:27.556409 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:27.556414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:27.556418 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:27.556596 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:27.557088 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:27.557095 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:27.557415 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 15:03:27.557627 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 15:03:27.558077 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":271,"threads_started":1}
I20260808 15:03:27.560384 15511 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46055
I20260808 15:03:27.560660 15511 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:27.561314 15511 heartbeater.cc:511] Master 127.2.180.254:46055 requested a full tablet report, sending...
I20260808 15:03:27.562585 15018 ts_manager.cc:194] Registered new tserver with Master: 8665747ff23c477ba2156e8e4d6c23a0 (127.2.180.194:40587)
I20260808 15:03:27.563369 15018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:50467
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:03:27.596443 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46055
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:27.908838 15519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:27.909101 15519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:27.909238 15519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:27.916190 15519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:27.916289 15519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:27.916440 15519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:27.925611 15519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:27.926759 15519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:27.927985 15519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:27.940132 15526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:27.940114 15529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:27.940132 15527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:27.940915 15519 server_base.cc:1034] running on GCE node
I20260808 15:03:27.941447 15519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:27.942281 15519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:27.943481 15519 hybrid_clock.cc:648] HybridClock initialized: now 1786201407943433 us; error 37 us; skew 500 ppm
I20260808 15:03:27.943763 15519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:27 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:27.946650 15519 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:03:27.947926 15519 webserver.cc:460] Webserver started at http://127.2.180.195:43339/ using document root <none> and password file <none>
I20260808 15:03:27.948529 15519 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:27.948642 15519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:27.948916 15519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:27.951597 15519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "df07d2aeb9604725af86295cba100588"
format_stamp: "Formatted at 2026-08-08 15:03:27 on dist-test-slave-jlzn"
server_key: "e04b1bff36657aa86a6c9e2ad93c867a"
server_key_iv: "255bbc732673db3082db024e7f73d8b9"
server_key_version: "encryptionkey@0"
I20260808 15:03:27.952406 15519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "df07d2aeb9604725af86295cba100588"
format_stamp: "Formatted at 2026-08-08 15:03:27 on dist-test-slave-jlzn"
server_key: "e04b1bff36657aa86a6c9e2ad93c867a"
server_key_iv: "255bbc732673db3082db024e7f73d8b9"
server_key_version: "encryptionkey@0"
I20260808 15:03:27.957093 15519 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 15:03:27.960269 15539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:27.961474 15519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:03:27.961630 15519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "df07d2aeb9604725af86295cba100588"
format_stamp: "Formatted at 2026-08-08 15:03:27 on dist-test-slave-jlzn"
server_key: "e04b1bff36657aa86a6c9e2ad93c867a"
server_key_iv: "255bbc732673db3082db024e7f73d8b9"
server_key_version: "encryptionkey@0"
I20260808 15:03:27.961807 15519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:27.990808 15519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:27.994712 15519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:27.994951 15519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:27.995872 15519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:27.997062 15519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:27.997135 15519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:27.997238 15519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:27.997298 15519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:28.036330 15519 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:43173
I20260808 15:03:28.036396 15723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:43173 every 8 connection(s)
I20260808 15:03:28.037568 15519 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:28.046376 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 15519
I20260808 15:03:28.046521 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:03:28.046974 2771 external_mini_cluster.cc:1428] Setting key ca6131d51c4f50824046b400f316ac50
Aug 08 15:03:28 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201407, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:28.056967 15315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:28.040009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:59511 (local address 127.2.180.254:46055)
0808 15:03:28.040195 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:28.040200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:28.041353 (+ 1153us) server_negotiation.cc:407] Connection header received
0808 15:03:28.043494 (+ 2141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:28.043500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:28.043590 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:28.043678 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:28.045172 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.045906 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.047546 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.048090 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.048217 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:28.051691 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:28.051696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:28.051699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:28.051733 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:28.054126 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:28.054791 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:28.054797 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:28.054801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:28.054925 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:28.055338 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:28.055343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:28.055347 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:28.055608 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:28.055738 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:28.056407 (+ 669us) server_negotiation.cc:299] Negotiation successful
0808 15:03:28.056632 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 15:03:28.057821 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:28.040465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:59511)
0808 15:03:28.041196 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:28.041237 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:28.043197 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:28.043950 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:28.043958 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:28.044446 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:28.045008 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:28.045026 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.046079 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.046082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:28.047391 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:28.047406 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.048308 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.048315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:28.048563 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:28.049200 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:28.049223 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:28.051447 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:28.054262 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:28.054270 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:28.054285 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:28.054679 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:28.055033 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:28.055038 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:28.055043 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:28.055230 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:28.055728 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:28.055736 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:28.056072 (+ 336us) client_negotiation.cc:769] Sending connection context
0808 15:03:28.056297 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 15:03:28.056776 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":273,"threads_started":1}
I20260808 15:03:28.059043 15724 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46055
I20260808 15:03:28.059329 15724 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:28.060070 15724 heartbeater.cc:511] Master 127.2.180.254:46055 requested a full tablet report, sending...
I20260808 15:03:28.061589 15018 ts_manager.cc:194] Registered new tserver with Master: df07d2aeb9604725af86295cba100588 (127.2.180.195:43173)
I20260808 15:03:28.062387 15018 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:59511
I20260808 15:03:28.073679 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.193@KRBTEST.COM" created.
I20260808 15:03:28.097858 15312 heartbeater.cc:503] Master 127.2.180.254:46055 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab.
Entry for principal oryx/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab.
WARNING: no policy specified for oryx/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab.
Entry for principal oryx/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab.
WARNING: no policy specified for oryx/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab.
Entry for principal oryx/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab.
I20260808 15:03:28.200156 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 15116
Aug 08 15:03:28 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:03:28.215296 15740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:28.201266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45056 (local address 127.2.180.193:33981)
0808 15:03:28.201903 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:28.201911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:28.201938 (+ 27us) server_negotiation.cc:407] Connection header received
0808 15:03:28.202004 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:28.202010 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:28.202231 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:28.202361 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:28.203700 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.204742 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.205894 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.206426 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.206517 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:28.210170 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:28.210180 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:28.210192 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:28.210230 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:28.213073 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:28.213631 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:28.213638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:28.213640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:28.213770 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:28.214034 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:28.214037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:28.214038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:28.214362 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:28.214598 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:28.214822 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 15:03:28.215011 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":315,"threads_started":1}
I20260808 15:03:28.216585 15306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:28.297482 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 15116
I20260808 15:03:28.317719 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 15324
Aug 08 15:03:28 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:03:28.333159 15743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:28.318611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43692 (local address 127.2.180.194:40587)
0808 15:03:28.319111 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:28.319118 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:28.319144 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:03:28.319214 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:28.319221 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:28.319427 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:28.319567 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:28.320874 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.321858 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.323109 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.323663 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.323773 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:28.327906 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:28.327917 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:28.327931 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:28.327962 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:28.330466 (+ 2504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:28.331110 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:28.331117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:28.331120 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:28.331261 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:28.331606 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:28.331612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:28.331615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:28.331993 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:28.332301 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:28.332568 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 15:03:28.332822 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":256,"threads_started":1}
I20260808 15:03:28.334347 15503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:28.414085 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 15324
I20260808 15:03:28.430186 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 15519
Aug 08 15:03:28 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:03:28.445495 15747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:28.431179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.2.180.195:43173)
0808 15:03:28.431855 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:28.431864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:28.431900 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:03:28.431979 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:28.431985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:28.432213 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:28.432337 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:28.433528 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.434511 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.435758 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:28.436354 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:28.436451 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:28.440490 (+ 4039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:28.440501 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:28.440516 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:28.440547 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:28.443220 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:28.443737 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:28.443744 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:28.443746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:28.443910 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:28.444210 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:28.444213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:28.444215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:28.444528 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:28.444763 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:28.444961 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 15:03:28.445163 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":335,"threads_started":1}
I20260808 15:03:28.446662 15716 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:28.526548 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 15519
I20260808 15:03:28.549490 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 14967
I20260808 15:03:28.550645 15070 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:28.651731 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 14967
I20260808 15:03:28.678615 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46055
--webserver_interface=127.2.180.254
--webserver_port=35871
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:46055
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:28.994482 15751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:28.994748 15751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:28.994875 15751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:29.001806 15751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:29.001901 15751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:29.001972 15751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:29.002044 15751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:29.002103 15751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:29.011314 15751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:46055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=35871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:29.012511 15751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:29.013748 15751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:29.025194 15761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:29.025291 15764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:29.025491 15767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:29.026055 15751 server_base.cc:1034] running on GCE node
I20260808 15:03:29.026767 15751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:29.028060 15751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:29.029264 15751 hybrid_clock.cc:648] HybridClock initialized: now 1786201409029215 us; error 41 us; skew 500 ppm
I20260808 15:03:29.029561 15751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:29 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201409, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:29.032387 15751 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:03:29.033702 15751 webserver.cc:460] Webserver started at http://127.2.180.254:35871/ using document root <none> and password file <none>
I20260808 15:03:29.034317 15751 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:29.034451 15751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:29.039498 15751 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:03:29.042078 15782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:29.043201 15751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:03:29.043347 15751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9"
format_stamp: "Formatted at 2026-08-08 15:03:26 on dist-test-slave-jlzn"
server_key: "f85f7112e37ec204673f4db22b115d0c"
server_key_iv: "a54d9ecaa184a7dcadb22b566df8c70e"
server_key_version: "encryptionkey@0"
I20260808 15:03:29.044299 15751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:29.067728 15751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:29.071260 15751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:29.071491 15751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:29.088301 15751 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:46055
I20260808 15:03:29.088325 15856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:46055 every 8 connection(s)
I20260808 15:03:29.089643 15751 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:29.096138 15857 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:03:29.098234 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 15751
I20260808 15:03:29.099011 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:33981
--local_ip_for_outbound_sockets=127.2.180.193
--tserver_master_addrs=127.2.180.254:46055
--webserver_port=36573
--webserver_interface=127.2.180.193
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.180.193
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:03:29.099184 15857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Bootstrap starting.
I20260808 15:03:29.120082 15857 log.cc:824] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:29.131167 15857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:03:29.131780 15857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Bootstrap complete.
I20260808 15:03:29.144809 15857 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } }
I20260808 15:03:29.145903 15857 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce5a8555a7584ae8ab3a3a2dde2010e9, State: Initialized, Role: FOLLOWER
I20260808 15:03:29.146585 15857 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } }
I20260808 15:03:29.146857 15857 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:29.146973 15857 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:29.147164 15857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:03:29.152536 15857 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } }
I20260808 15:03:29.153091 15857 leader_election.cc:302] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce5a8555a7584ae8ab3a3a2dde2010e9; no voters:
I20260808 15:03:29.153661 15857 leader_election.cc:288] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 15:03:29.153767 15862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:03:29.154850 15862 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [term 2 LEADER]: Becoming Leader. State: Replica: ce5a8555a7584ae8ab3a3a2dde2010e9, State: Running, Role: LEADER
I20260808 15:03:29.155359 15862 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } }
I20260808 15:03:29.155642 15857 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:29.157666 15864 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce5a8555a7584ae8ab3a3a2dde2010e9. Latest consensus state: current_term: 2 leader_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } } }
I20260808 15:03:29.158044 15864 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:29.157893 15863 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce5a8555a7584ae8ab3a3a2dde2010e9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46055 } } }
I20260808 15:03:29.158190 15863 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:29.168890 15868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:29.174436 15868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:29.175782 15868 catalog_manager.cc:1259] Loaded cluster ID: ab85f4890dba4f29918298ee3ffb7fcb
I20260808 15:03:29.175927 15868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:29.179730 15868 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:29.181239 15868 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ce5a8555a7584ae8ab3a3a2dde2010e9: Loaded TSK: 0
I20260808 15:03:29.182698 15868 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:03:29.470541 15860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:29.470824 15860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:29.470958 15860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:29.477921 15860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:29.478029 15860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:29.478173 15860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:29.487317 15860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.193.keytab
--principal=oryx/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:33981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=36573
--tserver_master_addrs=127.2.180.254:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:29.488545 15860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:29.489753 15860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:29.503643 15899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:29.503677 15900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:29.504388 15903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:29.505681 15860 server_base.cc:1034] running on GCE node
I20260808 15:03:29.506273 15860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:29.507143 15860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:29.508345 15860 hybrid_clock.cc:648] HybridClock initialized: now 1786201409508297 us; error 41 us; skew 500 ppm
I20260808 15:03:29.508632 15860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:29 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201409, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:29.511689 15860 init.cc:377] Logged in from keytab as oryx/127.2.180.193@KRBTEST.COM (short username oryx)
I20260808 15:03:29.513170 15860 webserver.cc:460] Webserver started at http://127.2.180.193:36573/ using document root <none> and password file <none>
I20260808 15:03:29.513820 15860 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:29.513948 15860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:29.519095 15860 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:03:29.521625 15922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:29.522929 15860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 15:03:29.523072 15860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "66e83c71940b4b9681b9ca1d960d2709"
format_stamp: "Formatted at 2026-08-08 15:03:26 on dist-test-slave-jlzn"
server_key: "d56a563287e7856dcb0a3f20083dfc4b"
server_key_iv: "f6e7b55561159ceab86588150a88a521"
server_key_version: "encryptionkey@0"
I20260808 15:03:29.523988 15860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:29.541778 15860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:29.545624 15860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:29.545882 15860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:29.546766 15860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:29.548079 15860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:29.548193 15860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:29.548301 15860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:29.548375 15860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:29.577961 15860 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:33981
I20260808 15:03:29.578050 16093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:33981 every 8 connection(s)
I20260808 15:03:29.579288 15860 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:29.586962 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 15860
I20260808 15:03:29.587678 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:40587
--local_ip_for_outbound_sockets=127.2.180.194
--tserver_master_addrs=127.2.180.254:46055
--webserver_port=38513
--webserver_interface=127.2.180.194
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.180.194
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:03:29 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:29 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 15:03:29.905990 16100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:29.906230 16100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:29.906340 16100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:29.913179 16100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:29.913270 16100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:29.913415 16100 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:29.922410 16100 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.194.keytab
--principal=oryx/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:40587
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=38513
--tserver_master_addrs=127.2.180.254:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:29.923522 16100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:29.924731 16100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:29.939301 16111 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:29.939328 16109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:29.940065 16108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:29.940398 16100 server_base.cc:1034] running on GCE node
I20260808 15:03:29.941155 16100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:29.942312 16100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:29.943552 16100 hybrid_clock.cc:648] HybridClock initialized: now 1786201409943486 us; error 48 us; skew 500 ppm
I20260808 15:03:29.943965 16100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:29 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201409, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:29.952140 16100 init.cc:377] Logged in from keytab as oryx/127.2.180.194@KRBTEST.COM (short username oryx)
I20260808 15:03:29.953884 16100 webserver.cc:460] Webserver started at http://127.2.180.194:38513/ using document root <none> and password file <none>
I20260808 15:03:29.954815 16100 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:29.954974 16100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:29.962276 16100 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:03:29.965677 16120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:29.967278 16100 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:03:29.967466 16100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "8665747ff23c477ba2156e8e4d6c23a0"
format_stamp: "Formatted at 2026-08-08 15:03:27 on dist-test-slave-jlzn"
server_key: "e225c83d77741da4c947d862ad5d78b3"
server_key_iv: "b81f7c726f1c5240a8b80cb4bdf9d80e"
server_key_version: "encryptionkey@0"
I20260808 15:03:29.968971 16100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:29.999198 16100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:30.004492 16100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:30.004796 16100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:30.005971 16100 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:30.007732 16100 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:30.007875 16100 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 15:03:30.007997 16100 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:30.008090 16100 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:30.042279 16100 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:40587
I20260808 15:03:30.042296 16293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:40587 every 8 connection(s)
I20260808 15:03:30.044231 16100 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:30.051710 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 16100
I20260808 15:03:30.052449 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:43173
--local_ip_for_outbound_sockets=127.2.180.195
--tserver_master_addrs=127.2.180.254:46055
--webserver_port=43339
--webserver_interface=127.2.180.195
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.2.180.195
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 15:03:30.377653 16298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:30.377914 16298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:30.378069 16298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:30.384920 16298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:30.385028 16298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:30.385180 16298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:30.394287 16298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/krb5kdc/oryx_127.2.180.195.keytab
--principal=oryx/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:43173
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=43339
--tserver_master_addrs=127.2.180.254:46055
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:30.395498 16298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:30.396755 16298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:30.411459 16311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:30.420189 16305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:30.421700 16313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:30.422477 16298 server_base.cc:1034] running on GCE node
I20260808 15:03:30.423321 16298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:30.424665 16298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:30.425987 16298 hybrid_clock.cc:648] HybridClock initialized: now 1786201410425881 us; error 94 us; skew 500 ppm
I20260808 15:03:30.426393 16298 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201410, etypes {rep=17 tkt=17 ses=17}, oryx/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:30.431155 16298 init.cc:377] Logged in from keytab as oryx/127.2.180.195@KRBTEST.COM (short username oryx)
I20260808 15:03:30.433318 16298 webserver.cc:460] Webserver started at http://127.2.180.195:43339/ using document root <none> and password file <none>
I20260808 15:03:30.434314 16298 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:30.434482 16298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:30.442343 16298 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 15:03:30.446033 16326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:30.447865 16298 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 15:03:30.448053 16298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "df07d2aeb9604725af86295cba100588"
format_stamp: "Formatted at 2026-08-08 15:03:27 on dist-test-slave-jlzn"
server_key: "e04b1bff36657aa86a6c9e2ad93c867a"
server_key_iv: "255bbc732673db3082db024e7f73d8b9"
server_key_version: "encryptionkey@0"
I20260808 15:03:30.449515 16298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:30.494223 16298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:30.500054 16298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:30.500379 16298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:30.501685 16298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:30.503492 16298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:30.503626 16298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:30.503748 16298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:30.503844 16298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:30.548020 16298 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:43173
I20260808 15:03:30.548112 16501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:43173 every 8 connection(s)
I20260808 15:03:30.550137 16298 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:30.550818 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 16298
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:30.603790 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:30.552259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57544 (local address 127.2.180.254:46055)
0808 15:03:30.577249 (+ 24990us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:30.577257 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:30.580117 (+ 2860us) server_negotiation.cc:407] Connection header received
0808 15:03:30.580203 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:30.580209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:30.580300 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:30.580402 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:30.583054 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.584409 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.588136 (+ 3727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.588847 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.588948 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:30.593082 (+ 4134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:30.593096 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:30.593118 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:30.593170 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:30.597943 (+ 4773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:30.598907 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:30.598918 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:30.598928 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:30.599164 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:30.599654 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:30.599660 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:30.599663 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:30.600347 (+ 684us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:30.600698 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:30.602145 (+ 1447us) server_negotiation.cc:299] Negotiation successful
0808 15:03:30.602653 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24850,"thread_start_us":24996,"threads_started":1}
W20260808 15:03:30.808363 16099 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.216s user 0.010s sys 0.003s
W20260808 15:03:30.810283 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:29.582035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:39503 (local address 127.2.180.254:46055)
0808 15:03:29.582756 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:29.582770 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:29.583033 (+ 263us) server_negotiation.cc:407] Connection header received
0808 15:03:29.585081 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:29.585094 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:29.585389 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:29.585692 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:29.587486 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:29.588791 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:29.590812 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:29.591415 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:29.591556 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:30.809872 (+1218316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:39503: BlockingRecv error: recv got EOF from 127.2.180.193:39503 (error 108)
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":245,"threads_started":1}
I20260808 15:03:30.810891 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:29.582070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:39503)
0808 15:03:29.582865 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:29.582905 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:29.584557 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:29.585866 (+ 1309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:29.585878 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:29.586552 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:29.587279 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:29.587306 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:29.588970 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:29.588975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:29.590685 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:29.590700 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:29.591655 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:29.591662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:29.591944 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:29.592582 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:29.592613 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:30.809491 (+1216878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":324,"threads_started":1}
W20260808 15:03:30.812443 16094 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:46055 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.180.254:46055: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:30.828167 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:30.813286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:53847)
0808 15:03:30.814525 (+ 1239us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:30.814548 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:30.814748 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:30.815320 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:30.815326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:30.815936 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:30.816206 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:30.816221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.817877 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.817884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:30.819639 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:30.819656 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.820964 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.820971 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:30.821190 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:30.821898 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:30.821925 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:30.827720 (+ 5795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1091,"thread_start_us":8292,"threads_started":1}
W20260808 15:03:30.828310 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:30.813675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:53847 (local address 127.2.180.254:46055)
0808 15:03:30.813938 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:30.813945 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:30.814625 (+ 680us) server_negotiation.cc:407] Connection header received
0808 15:03:30.814895 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:30.814901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:30.814986 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:30.815057 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:30.816399 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.817626 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.819847 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.820537 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.820638 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:30.828019 (+ 7381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:53847: BlockingRecv error: recv got EOF from 127.2.180.193:53847 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:30 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:30.841310 16099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:30.829450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:33299)
0808 15:03:30.829690 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:30.829710 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:30.829811 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:30.830188 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:30.830193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:30.830631 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:30.830847 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:30.830859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.832359 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.832364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:30.834070 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:30.834087 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.835154 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.835161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:30.835343 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:30.836115 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:30.836342 (+ 227us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:30.840995 (+ 4653us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128}
W20260808 15:03:30.841553 16098 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:30.829701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:33299 (local address 127.2.180.254:46055)
0808 15:03:30.829874 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:30.829881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:30.829901 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:03:30.829956 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:30.829962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:30.830041 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:30.830158 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:30.830988 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.832215 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.834220 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.834988 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.835116 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:30.841236 (+ 6120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:33299: BlockingRecv error: recv got EOF from 127.2.180.193:33299 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 15:03:30.841810 16094 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:03:31.190990 16297 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.133s user 0.004s sys 0.011s
W20260808 15:03:31.192466 16296 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:30.046977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:52817 (local address 127.2.180.254:46055)
0808 15:03:30.047430 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:30.047434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:30.048686 (+ 1252us) server_negotiation.cc:407] Connection header received
0808 15:03:30.050449 (+ 1763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:30.050456 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:30.050531 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:30.050624 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:30.052474 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.053764 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.055792 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.056525 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.056702 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.192233 (+1135531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:52817: BlockingRecv error: recv got EOF from 127.2.180.194:52817 (error 108)
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":211,"threads_started":1}
I20260808 15:03:31.193441 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:30.047674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:52817)
0808 15:03:30.048529 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:30.048575 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:30.050191 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:30.050869 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:30.050876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:30.051471 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:30.052270 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:30.052295 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.053955 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.053961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:30.055665 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:30.055681 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.057075 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.057083 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:30.057396 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:30.058119 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:30.058157 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:31.192091 (+1133934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":328,"threads_started":1}
W20260808 15:03:31.194409 16294 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:46055 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.180.254:46055: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:31.204248 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:31.195023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:37341)
0808 15:03:31.195249 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:31.195266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:31.195354 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:31.195677 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:31.195680 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:31.196148 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:31.196339 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.196352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.198062 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.198066 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.198989 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.199001 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.199963 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.199967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.200095 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.200753 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:31.200769 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:31.204053 (+ 3284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 15:03:31.204578 16296 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:31.195099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:37341 (local address 127.2.180.254:46055)
0808 15:03:31.195289 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:31.195296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:31.195318 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:03:31.195461 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:31.195467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:31.195559 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:31.195670 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:31.196534 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.197908 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.199116 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.199783 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.199937 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.204303 (+ 4366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:37341: BlockingRecv error: recv got EOF from 127.2.180.194:37341 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:31.213997 16297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:31.205005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:36727)
0808 15:03:31.205222 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:31.205244 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:31.205346 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:31.205638 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:31.205643 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:31.206066 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:31.206276 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.206287 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.207305 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.207311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.208954 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.208970 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.209665 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.209671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.209862 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.210398 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:31.210422 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:31.213829 (+ 3407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260808 15:03:31.214350 16296 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:31.205224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:36727 (local address 127.2.180.254:46055)
0808 15:03:31.205373 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:31.205379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:31.205397 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:03:31.205441 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:31.205444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:31.205494 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:31.205567 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:31.206412 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.207164 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.209141 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.209519 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.209613 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.214086 (+ 4473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:36727: BlockingRecv error: recv got EOF from 127.2.180.194:36727 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 15:03:31.214421 16294 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:03:31.608937 16506 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.012s user 0.000s sys 0.019s
W20260808 15:03:31.610030 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:30.577423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:45969 (local address 127.2.180.254:46055)
0808 15:03:30.587506 (+ 10083us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:30.587521 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:30.587539 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:03:30.587594 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:30.587599 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:30.587674 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:30.587792 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:30.590399 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.591773 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.594448 (+ 2675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.595212 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.595334 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.609824 (+1014490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:45969: BlockingRecv error: recv got EOF from 127.2.180.195:45969 (error 108)
Metrics: {"server-negotiator.queue_time_us":9927,"thread_start_us":10542,"threads_started":1}
I20260808 15:03:31.610781 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:30.559972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:45969)
0808 15:03:30.572426 (+ 12454us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:30.572480 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:30.575069 (+ 2589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:30.588376 (+ 13307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:30.588389 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:30.589248 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:30.590142 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:30.590169 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.591995 (+ 1826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.592001 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:30.594244 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:30.594263 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:30.595672 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:30.595679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:30.596016 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:30.596721 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:30.596753 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:31.609672 (+1012919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":12090,"thread_start_us":15976,"threads_started":1}
W20260808 15:03:31.611658 16503 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:46055 (0 consecutive failures): Not authorized: Failed to ping master at 127.2.180.254:46055: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:31.620325 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:31.612285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:43441)
0808 15:03:31.612477 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:31.612493 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:31.612593 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:31.612853 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:31.612855 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:31.613209 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:31.613365 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.613374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.614307 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.614310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.615237 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.615249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.615878 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.615883 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.616011 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.616623 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:31.616641 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:31.620129 (+ 3488us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 15:03:31.620481 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:31.612359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:43441 (local address 127.2.180.254:46055)
0808 15:03:31.612514 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:31.612518 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:31.612538 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:03:31.612687 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:31.612690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:31.612750 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:31.612826 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:31.613468 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.614203 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.615337 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.615721 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.615850 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.620302 (+ 4452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:43441: BlockingRecv error: recv got EOF from 127.2.180.195:43441 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:31.628294 16506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:31.620883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:34865)
0808 15:03:31.621003 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:31.621017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:31.621104 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:31.621475 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:31.621480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:31.621761 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:31.621902 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.621910 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.622859 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.622863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.623839 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.623854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.624542 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.624547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.624695 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.625165 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:31.625181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:31.628121 (+ 2940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 15:03:31.628535 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:31.620986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:34865 (local address 127.2.180.254:46055)
0808 15:03:31.621245 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:31.621250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:31.621264 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:03:31.621298 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:31.621301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:31.621347 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:31.621421 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:31.622032 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.622729 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.623982 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.624405 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.624494 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.628299 (+ 3805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:34865: BlockingRecv error: recv got EOF from 127.2.180.195:34865 (error 108)
Metrics: {"server-negotiator.queue_time_us":150}
W20260808 15:03:31.628549 16503 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:31 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:31.850919 16517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:31.842324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:50143)
0808 15:03:31.842862 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:31.842877 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:31.842990 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:31.843257 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:31.843260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:31.843693 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:31.843865 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.843875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.844754 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.844757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.845711 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:31.845723 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.846345 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.846349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:31.846470 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.847008 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:31.847026 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:31.850602 (+ 3576us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":319,"threads_started":1}
W20260808 15:03:31.851027 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:31.842398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:50143 (local address 127.2.180.254:46055)
0808 15:03:31.842558 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:31.842563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:31.842940 (+ 377us) server_negotiation.cc:407] Connection header received
0808 15:03:31.843074 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:31.843077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:31.843151 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:31.843229 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:31.843969 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.844637 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.845812 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:31.846196 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:31.846277 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:31.850821 (+ 4544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:50143: BlockingRecv error: recv got EOF from 127.2.180.193:50143 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 15:03:32 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:32 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:32.223647 16518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:32.214996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:55983)
0808 15:03:32.215506 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:32.215523 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:32.215638 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:32.215968 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:32.215972 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:32.216546 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:32.216717 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:32.216730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.217904 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.217907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:32.218868 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:32.218879 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.219522 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.219525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:32.219642 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:32.220243 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:32.220260 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:32.223413 (+ 3153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":286,"threads_started":1}
W20260808 15:03:32.223847 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:32.215052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:55983 (local address 127.2.180.254:46055)
0808 15:03:32.215230 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:32.215238 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:32.215592 (+ 354us) server_negotiation.cc:407] Connection header received
0808 15:03:32.215740 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:32.215743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:32.215851 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:32.215952 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:32.216884 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.217779 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.218972 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.219394 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.219477 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:32.223599 (+ 4122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:55983: BlockingRecv error: recv got EOF from 127.2.180.194:55983 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 15:03:32 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:32 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:32.637868 16519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:32.629150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:55775)
0808 15:03:32.629732 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:32.629752 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:32.629886 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:32.630239 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:32.630243 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:32.630781 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:32.630935 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:32.630945 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.631947 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.631953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:32.632961 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:32.632974 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.633663 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.633669 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:32.633796 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:32.634291 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:32.634307 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:32.637623 (+ 3316us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":329,"threads_started":1}
W20260808 15:03:32.638010 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:32.629266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:55775 (local address 127.2.180.254:46055)
0808 15:03:32.629464 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:32.629470 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:32.629831 (+ 361us) server_negotiation.cc:407] Connection header received
0808 15:03:32.630003 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:32.630008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:32.630095 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:32.630191 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:32.631058 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.631793 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.633076 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.633522 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.633611 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:32.637799 (+ 4188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:55775: BlockingRecv error: recv got EOF from 127.2.180.195:55775 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 15:03:32 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:32 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:32.860579 16520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:32.851936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:55443)
0808 15:03:32.852374 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:32.852390 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:32.852510 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:32.852778 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:32.852781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:32.853161 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:32.853349 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:32.853357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.854327 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.854330 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:32.855324 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:32.855338 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.856107 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.856114 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:32.856303 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:32.856838 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:32.856855 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:32.860249 (+ 3394us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":233,"threads_started":1}
W20260808 15:03:32.860728 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:32.851951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:55443 (local address 127.2.180.254:46055)
0808 15:03:32.852167 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:32.852174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:32.852459 (+ 285us) server_negotiation.cc:407] Connection header received
0808 15:03:32.852604 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:32.852607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:32.852671 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:32.852748 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:32.853463 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.854218 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.855455 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:32.855950 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:32.856069 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:32.860479 (+ 4410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:55443: BlockingRecv error: recv got EOF from 127.2.180.193:55443 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 15:03:33 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:33 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:33.232806 16522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:33.224491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:51053)
0808 15:03:33.224986 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:33.225014 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:33.225123 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:33.225371 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:33.225374 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:33.225773 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:33.225959 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:33.225968 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.226916 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.226920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:33.227922 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:33.227939 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.228676 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.228681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:33.228811 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:33.229340 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:33.229359 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:33.232557 (+ 3198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":262,"threads_started":1}
W20260808 15:03:33.232975 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:33.224551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:51053 (local address 127.2.180.254:46055)
0808 15:03:33.224717 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:33.224722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:33.225074 (+ 352us) server_negotiation.cc:407] Connection header received
0808 15:03:33.225204 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:33.225207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:33.225265 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:33.225342 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:33.226073 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.226790 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.228051 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.228507 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.228599 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:33.232733 (+ 4134us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:51053: BlockingRecv error: recv got EOF from 127.2.180.194:51053 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 15:03:33 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:33 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:33.646708 16524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:33.638702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:34527)
0808 15:03:33.639192 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:33.639208 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:33.639319 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:33.639599 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:33.639601 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:33.640019 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:33.640213 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:33.640224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.641169 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.641173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:33.642112 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:33.642125 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.642701 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.642705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:33.642829 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:33.643331 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:33.643348 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:33.646437 (+ 3089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":256,"threads_started":1}
W20260808 15:03:33.646811 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:33.638801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:34527 (local address 127.2.180.254:46055)
0808 15:03:33.638949 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:33.638953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:33.639268 (+ 315us) server_negotiation.cc:407] Connection header received
0808 15:03:33.639429 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:33.639432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:33.639502 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:33.639573 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:33.640334 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.641047 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.642216 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.642591 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.642682 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:33.646610 (+ 3928us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:34527: BlockingRecv error: recv got EOF from 127.2.180.195:34527 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 15:03:33 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:33 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:33.871125 16525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:33.861714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:36861)
0808 15:03:33.863115 (+ 1401us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:33.863130 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:33.863263 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:33.863551 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:33.863554 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:33.863947 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:33.864126 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:33.864136 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.865066 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.865069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:33.865937 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:33.865947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.866570 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.866574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:33.866710 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:33.867361 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:33.867385 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:33.870810 (+ 3425us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1297,"thread_start_us":1157,"threads_started":1}
W20260808 15:03:33.871299 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:33.861712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:36861 (local address 127.2.180.254:46055)
0808 15:03:33.861860 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:33.861864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:33.863232 (+ 1368us) server_negotiation.cc:407] Connection header received
0808 15:03:33.863368 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:33.863372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:33.863436 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:33.863510 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:33.864253 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.864971 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.866042 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:33.866463 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:33.866554 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:33.871054 (+ 4500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:36861: BlockingRecv error: recv got EOF from 127.2.180.193:36861 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 15:03:34 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:34 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:34.242209 16526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:34.233723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:53315)
0808 15:03:34.234243 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:34.234259 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:34.234369 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:34.234683 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:34.234686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:34.235122 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:34.235310 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:34.235321 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.236633 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.236637 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:34.237535 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:34.237547 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.238141 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.238144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:34.238268 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:34.238782 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:34.238798 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:34.241969 (+ 3171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":249,"threads_started":1}
W20260808 15:03:34.242455 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:34.233790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:53315 (local address 127.2.180.254:46055)
0808 15:03:34.233960 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:34.233965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:34.234319 (+ 354us) server_negotiation.cc:407] Connection header received
0808 15:03:34.234457 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:34.234461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:34.234523 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:34.234644 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:34.235462 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.236497 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.237637 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.238032 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.238125 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:34.242194 (+ 4069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:53315: BlockingRecv error: recv got EOF from 127.2.180.194:53315 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 15:03:34 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:34 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:34.656049 16527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:34.647543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:53753)
0808 15:03:34.648120 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:34.648143 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:34.648266 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:34.648614 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:34.648618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:34.649098 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:34.649322 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:34.649335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.650305 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.650309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:34.651216 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:34.651227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.651940 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.651944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:34.652062 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:34.652572 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:34.652589 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:34.655783 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":261,"threads_started":1}
W20260808 15:03:34.656257 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:34.647677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:53753 (local address 127.2.180.254:46055)
0808 15:03:34.648217 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:34.648222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:34.648244 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:03:34.648395 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:34.648400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:34.648478 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:34.648605 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:34.649467 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.650189 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.651366 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.651831 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.651932 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:34.656005 (+ 4073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:53753: BlockingRecv error: recv got EOF from 127.2.180.195:53753 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 15:03:34 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:34 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:34.883406 16528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:34.872045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:41251)
0808 15:03:34.875305 (+ 3260us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:34.875321 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:34.875432 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:34.875695 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:34.875698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:34.876173 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:34.876346 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:34.876356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.877309 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.877312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:34.878325 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:34.878337 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.878930 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.878934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:34.879056 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:34.879723 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:34.879747 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:34.883165 (+ 3418us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":3163,"thread_start_us":3022,"threads_started":1}
W20260808 15:03:34.883586 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:34.872114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:41251 (local address 127.2.180.254:46055)
0808 15:03:34.872319 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:34.872326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:34.875396 (+ 3070us) server_negotiation.cc:407] Connection header received
0808 15:03:34.875513 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:34.875516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:34.875586 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:34.875661 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:34.876467 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.877206 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.878432 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:34.878826 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:34.878911 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:34.883368 (+ 4457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:41251: BlockingRecv error: recv got EOF from 127.2.180.193:41251 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 15:03:35 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:35 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:35.254616 16532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:35.243088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:50131)
0808 15:03:35.243531 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:35.243546 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:35.243664 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:35.243947 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:35.243951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:35.244323 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:35.244489 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:35.244498 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.245400 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.245403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:35.246379 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:35.246390 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.247132 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.247138 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:35.247304 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:35.250953 (+ 3649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:35.250971 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:35.254248 (+ 3277us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":226,"threads_started":1}
W20260808 15:03:35.254668 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:35.243125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:50131 (local address 127.2.180.254:46055)
0808 15:03:35.243340 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:35.243346 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:35.243615 (+ 269us) server_negotiation.cc:407] Connection header received
0808 15:03:35.243753 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:35.243756 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:35.243840 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:35.243915 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:35.244590 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.245304 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.246484 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.246966 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.247085 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:35.254448 (+ 7363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:50131: BlockingRecv error: recv got EOF from 127.2.180.194:50131 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 15:03:35 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:35 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:35.665382 16533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:35.656897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:34445)
0808 15:03:35.657340 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:35.657356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:35.657466 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:35.657768 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:35.657771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:35.658161 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:35.658356 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:35.658365 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.659351 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.659357 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:35.660627 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:35.660643 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.661303 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.661309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:35.661479 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:35.662038 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:35.662055 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:35.665039 (+ 2984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":255,"threads_started":1}
W20260808 15:03:35.665504 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:35.656937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:34445 (local address 127.2.180.254:46055)
0808 15:03:35.657123 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:35.657130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:35.657423 (+ 293us) server_negotiation.cc:407] Connection header received
0808 15:03:35.657559 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:35.657563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:35.657646 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:35.657730 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:35.658464 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.659209 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.660761 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.661168 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.661279 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:35.665266 (+ 3987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:34445: BlockingRecv error: recv got EOF from 127.2.180.195:34445 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 15:03:35 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:35 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:35.892421 16534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:35.884273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:56941)
0808 15:03:35.884766 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:35.884783 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:35.884893 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:35.885148 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:35.885151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:35.885556 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:35.885722 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:35.885731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.886659 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.886662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:35.887563 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:35.887574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.888184 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.888188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:35.888306 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:35.888814 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:35.888830 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:35.892197 (+ 3367us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":249,"threads_started":1}
W20260808 15:03:35.892580 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:35.884349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:56941 (local address 127.2.180.254:46055)
0808 15:03:35.884500 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:35.884504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:35.884844 (+ 340us) server_negotiation.cc:407] Connection header received
0808 15:03:35.884976 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:35.884979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:35.885039 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:35.885115 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:35.885823 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.886534 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.887666 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:35.888074 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:35.888171 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:35.892382 (+ 4211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:56941: BlockingRecv error: recv got EOF from 127.2.180.193:56941 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 15:03:36 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:36 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:36.264246 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:36.255451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:36421)
0808 15:03:36.256021 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:36.256038 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:36.256161 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:36.256461 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:36.256465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:36.256889 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:36.257045 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:36.257053 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.258431 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.258437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:36.259467 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:36.259480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.260133 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.260136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:36.260251 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:36.260770 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:36.260787 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:36.263930 (+ 3143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":255,"threads_started":1}
W20260808 15:03:36.264355 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:36.255601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:36421 (local address 127.2.180.254:46055)
0808 15:03:36.255767 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:36.255771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:36.256108 (+ 337us) server_negotiation.cc:407] Connection header received
0808 15:03:36.256243 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:36.256246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:36.256341 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:36.256428 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:36.257177 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.258293 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.259590 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.260027 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.260116 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:36.264118 (+ 4002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:36421: BlockingRecv error: recv got EOF from 127.2.180.194:36421 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 15:03:36 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:36 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:36.674768 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:36.666295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:32795)
0808 15:03:36.666765 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:36.666781 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:36.666906 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:36.667233 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:36.667239 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:36.667664 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:36.667845 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:36.667858 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.668840 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.668845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:36.669777 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:36.669789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.670545 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.670551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:36.670720 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:36.671254 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:36.671271 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:36.674474 (+ 3203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":245,"threads_started":1}
W20260808 15:03:36.674912 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:36.666373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:32795 (local address 127.2.180.254:46055)
0808 15:03:36.666560 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:36.666565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:36.666854 (+ 289us) server_negotiation.cc:407] Connection header received
0808 15:03:36.667021 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:36.667026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:36.667095 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:36.667173 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:36.667975 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.668706 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.669905 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.670411 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.670516 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:36.674682 (+ 4166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:32795: BlockingRecv error: recv got EOF from 127.2.180.195:32795 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 08 15:03:36 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:36 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:36.901846 16538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:36.893268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:42825)
0808 15:03:36.893877 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:36.893897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:36.894048 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:36.894417 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:36.894423 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:36.894929 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:36.895096 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:36.895106 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.896130 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.896133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:36.897047 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:36.897059 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.897716 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.897720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:36.897840 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:36.898356 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:36.898373 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:36.901612 (+ 3239us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":235,"threads_started":1}
W20260808 15:03:36.902032 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:36.893337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:42825 (local address 127.2.180.254:46055)
0808 15:03:36.893570 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:36.893575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:36.893982 (+ 407us) server_negotiation.cc:407] Connection header received
0808 15:03:36.894181 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:36.894186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:36.894270 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:36.894351 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:36.895229 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.896015 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.897166 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:36.897596 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:36.897688 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:36.901801 (+ 4113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:42825: BlockingRecv error: recv got EOF from 127.2.180.193:42825 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 08 15:03:37 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:37 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:37.273417 16540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:37.265142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:60785)
0808 15:03:37.265566 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:37.265582 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:37.265692 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:37.265953 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:37.265955 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:37.266375 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:37.266549 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:37.266557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.267438 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.267441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:37.268374 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:37.268386 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.269048 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.269053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:37.269183 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:37.269701 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:37.269718 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:37.273130 (+ 3412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":216,"threads_started":1}
W20260808 15:03:37.273468 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:37.265175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:60785 (local address 127.2.180.254:46055)
0808 15:03:37.265384 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:37.265390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:37.265654 (+ 264us) server_negotiation.cc:407] Connection header received
0808 15:03:37.265779 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:37.265783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:37.265849 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:37.265924 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:37.266653 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.267336 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.268478 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.268922 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.269011 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:37.273305 (+ 4294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:60785: BlockingRecv error: recv got EOF from 127.2.180.194:60785 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 15:03:37 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:37 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:37.683969 16542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:37.675576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:33185)
0808 15:03:37.676086 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:37.676102 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:37.676259 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:37.676575 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:37.676580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:37.676998 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:37.677170 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:37.677181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.678212 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.678218 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:37.679125 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:37.679137 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.679846 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.679854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:37.680027 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:37.680540 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:37.680557 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:37.683622 (+ 3065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":246,"threads_started":1}
W20260808 15:03:37.684108 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:37.675671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:33185 (local address 127.2.180.254:46055)
0808 15:03:37.675882 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:37.675889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:37.676184 (+ 295us) server_negotiation.cc:407] Connection header received
0808 15:03:37.676379 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:37.676385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:37.676447 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:37.676526 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:37.677313 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.678067 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.679276 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.679671 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.679773 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:37.683875 (+ 4102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:33185: BlockingRecv error: recv got EOF from 127.2.180.195:33185 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 15:03:37 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:37 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:37.911118 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:37.902699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:59353)
0808 15:03:37.903255 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:37.903272 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:37.903402 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:37.903735 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:37.903740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:37.904163 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:37.904333 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:37.904342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.905329 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.905333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:37.906268 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:37.906280 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.906990 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.906996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:37.907151 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:37.907677 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:37.907693 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:37.910779 (+ 3086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":261,"threads_started":1}
W20260808 15:03:37.911199 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:37.902774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:59353 (local address 127.2.180.254:46055)
0808 15:03:37.902952 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:37.902959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:37.903357 (+ 398us) server_negotiation.cc:407] Connection header received
0808 15:03:37.903509 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:37.903514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:37.903576 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:37.903656 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:37.904476 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.905192 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.906409 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:37.906847 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:37.906952 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:37.910982 (+ 4030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:59353: BlockingRecv error: recv got EOF from 127.2.180.193:59353 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 15:03:38 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:38 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:38.282891 16544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:38.274228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:47189)
0808 15:03:38.274930 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:38.274949 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:38.275109 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:38.275429 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:38.275434 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:38.275986 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:38.276162 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:38.276171 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.277141 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.277147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:38.278089 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:38.278100 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.278814 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.278819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:38.278983 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:38.279477 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:38.279494 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:38.282543 (+ 3049us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":358,"threads_started":1}
W20260808 15:03:38.283012 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:38.274311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:47189 (local address 127.2.180.254:46055)
0808 15:03:38.274497 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:38.274503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:38.275034 (+ 531us) server_negotiation.cc:407] Connection header received
0808 15:03:38.275236 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:38.275241 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:38.275305 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:38.275379 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:38.276302 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.277009 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.278229 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.278669 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.278776 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:38.282765 (+ 3989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:47189: BlockingRecv error: recv got EOF from 127.2.180.194:47189 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 08 15:03:38 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:38 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:38.693593 16545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:38.685018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:47043)
0808 15:03:38.685546 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:38.685562 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:38.685702 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:38.686034 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:38.686039 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:38.686467 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:38.686639 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:38.686658 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.687643 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.687654 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:38.688673 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:38.688691 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.689552 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.689559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:38.689717 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:38.690321 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:38.690339 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:38.693359 (+ 3020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":239,"threads_started":1}
W20260808 15:03:38.693730 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:38.685065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:47043 (local address 127.2.180.254:46055)
0808 15:03:38.685243 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:38.685249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:38.685646 (+ 397us) server_negotiation.cc:407] Connection header received
0808 15:03:38.685823 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:38.685828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:38.685889 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:38.685967 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:38.686798 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.687508 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.688820 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.689398 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.689507 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:38.693537 (+ 4030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:47043: BlockingRecv error: recv got EOF from 127.2.180.195:47043 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 15:03:38 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:38 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:38.920756 16547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:38.912073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:36201)
0808 15:03:38.912630 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:38.912646 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:38.912767 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:38.913022 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:38.913025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:38.913431 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:38.913598 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:38.913607 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.914608 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.914611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:38.915568 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:38.915581 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.916290 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.916294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:38.916402 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:38.916971 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:38.916990 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:38.920458 (+ 3468us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":277,"threads_started":1}
W20260808 15:03:38.920883 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:38.912233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:36201 (local address 127.2.180.254:46055)
0808 15:03:38.912403 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:38.912408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:38.912713 (+ 305us) server_negotiation.cc:407] Connection header received
0808 15:03:38.912851 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:38.912854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:38.912915 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:38.912992 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:38.913740 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.914500 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.915689 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:38.916149 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:38.916252 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:38.920675 (+ 4423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:36201: BlockingRecv error: recv got EOF from 127.2.180.193:36201 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 15:03:39 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:39 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:39.292682 16548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:39.283943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:51677)
0808 15:03:39.284455 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:39.284470 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:39.284593 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:39.284948 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:39.284953 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:39.285376 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:39.285542 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:39.285550 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.286666 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.286670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:39.287595 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:39.287607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.288529 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.288536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:39.288692 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:39.289262 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:39.289280 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:39.292431 (+ 3151us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":271,"threads_started":1}
W20260808 15:03:39.292866 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:39.283997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:51677 (local address 127.2.180.254:46055)
0808 15:03:39.284196 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:39.284202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:39.284548 (+ 346us) server_negotiation.cc:407] Connection header received
0808 15:03:39.284739 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:39.284744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:39.284816 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:39.284895 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:39.285678 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.286518 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.287734 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.288357 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.288470 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:39.292646 (+ 4176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:51677: BlockingRecv error: recv got EOF from 127.2.180.194:51677 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 15:03:39 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:39 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:39.703099 16550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:39.694421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:34235)
0808 15:03:39.694953 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:39.694968 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:39.695100 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:39.695412 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:39.695416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:39.695885 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:39.696051 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:39.696062 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.697068 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.697072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:39.698020 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:39.698032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.698765 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.698771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:39.698911 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:39.699494 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:39.699512 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:39.702767 (+ 3255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":296,"threads_started":1}
W20260808 15:03:39.703229 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:39.694499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:34235 (local address 127.2.180.254:46055)
0808 15:03:39.694685 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:39.694692 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:39.695051 (+ 359us) server_negotiation.cc:407] Connection header received
0808 15:03:39.695217 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:39.695220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:39.695280 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:39.695358 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:39.696211 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.696937 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.698164 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.698591 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.698695 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:39.702985 (+ 4290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:34235: BlockingRecv error: recv got EOF from 127.2.180.195:34235 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 15:03:39 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:39 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:39.930248 16551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:39.921784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:37821)
0808 15:03:39.922222 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:39.922238 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:39.922344 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:39.922613 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:39.922616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:39.923096 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:39.923252 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:39.923262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.924515 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.924519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:39.925482 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:39.925495 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.926116 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.926120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:39.926241 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:39.926787 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:39.926814 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:39.930006 (+ 3192us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
W20260808 15:03:39.930377 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:39.921836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:37821 (local address 127.2.180.254:46055)
0808 15:03:39.922073 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:39.922079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:39.922303 (+ 224us) server_negotiation.cc:407] Connection header received
0808 15:03:39.922427 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:39.922430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:39.922493 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:39.922594 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:39.923369 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.924378 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.925591 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:39.925999 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:39.926096 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:39.930188 (+ 4092us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:37821: BlockingRecv error: recv got EOF from 127.2.180.193:37821 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 15:03:40 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:40 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:40.307075 16553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:40.293645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:45935)
0808 15:03:40.294113 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:40.294129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:40.294260 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:40.294569 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:40.294574 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:40.295103 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:40.295294 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:40.295306 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.296375 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.296380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:40.297316 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:40.297327 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.298015 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.298021 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:40.298206 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:40.303610 (+ 5404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:40.303628 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:40.306844 (+ 3216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":227,"threads_started":1}
W20260808 15:03:40.307268 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:40.293701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:45935 (local address 127.2.180.254:46055)
0808 15:03:40.293893 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:40.293899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:40.294215 (+ 316us) server_negotiation.cc:407] Connection header received
0808 15:03:40.294373 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:40.294378 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:40.294442 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:40.294520 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:40.295441 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.296233 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.297455 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.297872 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.297980 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:40.307030 (+ 9050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:45935: BlockingRecv error: recv got EOF from 127.2.180.194:45935 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 15:03:40 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:40 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:40.712570 16557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:40.704115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:53193)
0808 15:03:40.704625 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:40.704641 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:40.704781 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:40.705101 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:40.705106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:40.705530 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:40.705677 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:40.705686 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.706649 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.706655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:40.707579 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:40.707591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.708388 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.708394 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:40.708559 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:40.709082 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:40.709099 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:40.712257 (+ 3158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":256,"threads_started":1}
W20260808 15:03:40.712682 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:40.704154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:53193 (local address 127.2.180.254:46055)
0808 15:03:40.704345 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:40.704352 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:40.704722 (+ 370us) server_negotiation.cc:407] Connection header received
0808 15:03:40.704900 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:40.704906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:40.704972 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:40.705048 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:40.705826 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.706522 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.707717 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.708242 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.708353 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:40.712464 (+ 4111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:53193: BlockingRecv error: recv got EOF from 127.2.180.195:53193 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 15:03:40 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:40 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:40.939400 16558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:40.931011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:54307)
0808 15:03:40.931483 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:40.931512 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:40.931640 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:40.931928 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:40.931931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:40.932302 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:40.932464 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:40.932473 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.933384 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.933388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:40.934366 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:40.934382 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.935144 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.935150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:40.935317 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:40.935936 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:40.935954 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:40.939177 (+ 3223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":244,"threads_started":1}
W20260808 15:03:40.939577 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:40.931099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:54307 (local address 127.2.180.254:46055)
0808 15:03:40.931260 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:40.931264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:40.931574 (+ 310us) server_negotiation.cc:407] Connection header received
0808 15:03:40.931726 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:40.931730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:40.931818 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:40.931896 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:40.932580 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.933287 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.934508 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:40.934991 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:40.935099 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:40.939341 (+ 4242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:54307: BlockingRecv error: recv got EOF from 127.2.180.193:54307 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 15:03:41 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:41 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:41.316923 16559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:41.307908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:58275)
0808 15:03:41.308357 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:41.308373 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:41.308498 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:41.308807 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:41.308812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:41.309207 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:41.309365 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:41.309374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.310344 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.310349 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:41.311336 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:41.311347 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.312099 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.312105 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:41.312363 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:41.313097 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:41.313115 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:41.316586 (+ 3471us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":224,"threads_started":1}
W20260808 15:03:41.317046 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:41.307958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:58275 (local address 127.2.180.254:46055)
0808 15:03:41.308138 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:41.308145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:41.308452 (+ 307us) server_negotiation.cc:407] Connection header received
0808 15:03:41.308608 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:41.308613 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:41.308679 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:41.308757 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:41.309511 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.310214 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.311473 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.311954 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.312061 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:41.316782 (+ 4721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:58275: BlockingRecv error: recv got EOF from 127.2.180.194:58275 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 08 15:03:41 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:41 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:41.722258 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:41.713470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:33685)
0808 15:03:41.713952 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:41.713969 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:41.714093 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:41.714385 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:41.714388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:41.714760 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:41.714932 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:41.714941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.715916 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.715920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:41.716895 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:41.716907 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.717569 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.717573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:41.717678 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:41.718294 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:41.718312 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:41.722001 (+ 3689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":259,"threads_started":1}
W20260808 15:03:41.722388 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:41.713636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:33685 (local address 127.2.180.254:46055)
0808 15:03:41.713851 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:41.713856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:41.714032 (+ 176us) server_negotiation.cc:407] Connection header received
0808 15:03:41.714195 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:41.714199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:41.714277 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:41.714356 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:41.715045 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.715761 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.717004 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.717436 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.717523 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:41.722187 (+ 4664us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:33685: BlockingRecv error: recv got EOF from 127.2.180.195:33685 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 08 15:03:41 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:41 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:41.949221 16564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:41.940266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:60705)
0808 15:03:41.940726 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:41.940742 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:41.940877 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:41.941233 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:41.941238 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:41.941687 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:41.941865 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:41.941875 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.942887 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.942892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:41.943836 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:41.943848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.944565 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.944571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:41.944731 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:41.945279 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:41.945296 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:41.948995 (+ 3699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":240,"threads_started":1}
W20260808 15:03:41.949383 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:41.940327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:60705 (local address 127.2.180.254:46055)
0808 15:03:41.940517 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:41.940523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:41.940829 (+ 306us) server_negotiation.cc:407] Connection header received
0808 15:03:41.940986 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:41.940991 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:41.941090 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:41.941173 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:41.942010 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.942748 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.943978 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:41.944409 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:41.944517 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:41.949184 (+ 4667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:60705: BlockingRecv error: recv got EOF from 127.2.180.193:60705 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 15:03:42 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:42 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:42.327373 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:42.317968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:32805)
0808 15:03:42.318438 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:42.318454 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:42.318578 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:42.318911 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:42.318916 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:42.319357 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:42.319529 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:42.319538 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.320611 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.320617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:42.321879 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:42.321897 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.322880 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.322884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:42.323003 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:42.323565 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:42.323582 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:42.327116 (+ 3534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":267,"threads_started":1}
W20260808 15:03:42.327611 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:42.318038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:32805 (local address 127.2.180.254:46055)
0808 15:03:42.318205 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:42.318211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:42.318528 (+ 317us) server_negotiation.cc:407] Connection header received
0808 15:03:42.318707 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:42.318713 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:42.318781 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:42.318857 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:42.319672 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.320484 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.322044 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.322732 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.322867 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:42.327340 (+ 4473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:32805: BlockingRecv error: recv got EOF from 127.2.180.194:32805 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 08 15:03:42 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:42 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:42.731066 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:42.723108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:42277)
0808 15:03:42.723514 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:42.723530 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:42.723638 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:42.723934 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:42.723937 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:42.724362 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:42.724583 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:42.724596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.725531 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.725534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:42.726436 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:42.726447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.727046 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.727050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:42.727180 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:42.727732 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:42.727749 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:42.730833 (+ 3084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":204,"threads_started":1}
W20260808 15:03:42.731223 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:42.723175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:42277 (local address 127.2.180.254:46055)
0808 15:03:42.723378 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:42.723384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:42.723596 (+ 212us) server_negotiation.cc:407] Connection header received
0808 15:03:42.723727 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:42.723730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:42.723794 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:42.723913 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:42.724707 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.725424 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.726532 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.726938 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.727022 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:42.731005 (+ 3983us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:42277: BlockingRecv error: recv got EOF from 127.2.180.195:42277 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 08 15:03:42 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:42 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:42.959102 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:42.950042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:39965)
0808 15:03:42.950615 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:42.950631 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:42.950746 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:42.951014 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:42.951017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:42.951413 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:42.951633 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:42.951643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.952713 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.952719 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:42.953928 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:42.953944 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.954613 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.954617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:42.954786 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:42.955368 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:42.955386 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:42.958851 (+ 3465us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":279,"threads_started":1}
W20260808 15:03:42.959251 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:42.950103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:39965 (local address 127.2.180.254:46055)
0808 15:03:42.950286 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:42.950290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:42.950694 (+ 404us) server_negotiation.cc:407] Connection header received
0808 15:03:42.950830 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:42.950833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:42.950906 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:42.950981 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:42.951744 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.952573 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.954063 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:42.954496 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:42.954585 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:42.959020 (+ 4435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:39965: BlockingRecv error: recv got EOF from 127.2.180.193:39965 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 15:03:43 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:43 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:43.336793 16571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:43.328497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:40421)
0808 15:03:43.328993 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:43.329010 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:43.329121 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:43.329398 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:43.329401 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:43.329800 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:43.329965 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:43.329972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.330904 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.330907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:43.331941 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:43.331956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.332654 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.332660 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:43.332832 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:43.333337 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:43.333354 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:43.336466 (+ 3112us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":261,"threads_started":1}
W20260808 15:03:43.336926 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:43.328523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:40421 (local address 127.2.180.254:46055)
0808 15:03:43.328730 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:43.328737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:43.329085 (+ 348us) server_negotiation.cc:407] Connection header received
0808 15:03:43.329217 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:43.329220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:43.329282 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:43.329356 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:43.330076 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.330801 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.332072 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.332483 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.332603 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:43.336680 (+ 4077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:40421: BlockingRecv error: recv got EOF from 127.2.180.194:40421 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 15:03:43 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:43 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:43.740726 16577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:43.731946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:42887)
0808 15:03:43.732494 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:43.732511 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:43.732645 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:43.732907 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:43.732910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:43.733296 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:43.733470 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:43.733480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.734486 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.734490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:43.735448 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:43.735462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.736221 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.736227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:43.736419 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:43.737085 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:43.737112 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:43.740470 (+ 3358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":305,"threads_started":1}
W20260808 15:03:43.740895 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:43.731995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:42887 (local address 127.2.180.254:46055)
0808 15:03:43.732149 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:43.732154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:43.732571 (+ 417us) server_negotiation.cc:407] Connection header received
0808 15:03:43.732737 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:43.732741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:43.732801 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:43.732878 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:43.733604 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.734379 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.735559 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.736066 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.736185 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:43.740670 (+ 4485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:42887: BlockingRecv error: recv got EOF from 127.2.180.195:42887 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 15:03:43 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:43 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:43.968040 16578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:43.959946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:47411)
0808 15:03:43.960413 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:43.960429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:43.960531 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:43.960780 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:43.960783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:43.961163 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:43.961311 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:43.961321 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.962240 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.962244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:43.963159 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:43.963172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.963760 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.963765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:43.963902 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:43.964467 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:43.964486 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:43.967680 (+ 3194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":224,"threads_started":1}
W20260808 15:03:43.968189 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:43.960369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:47411 (local address 127.2.180.254:46055)
0808 15:03:43.960528 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:43.960532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:43.960549 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:03:43.960601 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:43.960604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:43.960676 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:43.960751 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:43.961421 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.962131 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.963266 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:43.963656 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:43.963743 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:43.967922 (+ 4179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:47411: BlockingRecv error: recv got EOF from 127.2.180.193:47411 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 08 15:03:44 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:44 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:44.346351 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:44.337765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:40751)
0808 15:03:44.338229 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:44.338249 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:44.338387 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:44.338696 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:44.338701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:44.339144 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:44.339320 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:44.339334 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.340392 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.340396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:44.341320 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:44.341332 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.342034 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.342040 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:44.342213 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:44.342792 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:44.342811 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:44.346114 (+ 3303us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":227,"threads_started":1}
W20260808 15:03:44.346474 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:44.337917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:40751 (local address 127.2.180.254:46055)
0808 15:03:44.338126 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:44.338132 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:44.338328 (+ 196us) server_negotiation.cc:407] Connection header received
0808 15:03:44.338504 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:44.338508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:44.338569 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:44.338645 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:44.339477 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.340246 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.341454 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.341896 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.342002 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:44.346274 (+ 4272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:40751: BlockingRecv error: recv got EOF from 127.2.180.194:40751 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 15:03:44 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:44 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.195@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:44.749692 16580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:44.741548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.195:36461)
0808 15:03:44.741972 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:44.741988 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:44.742099 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:44.742378 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:44.742381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:44.742774 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:44.742919 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:44.742927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.743862 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.743865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:44.744743 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:44.744753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.745387 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.745390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:44.745504 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:44.746007 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:44.746024 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:44.749448 (+ 3424us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":222,"threads_started":1}
W20260808 15:03:44.749822 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:44.741620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:36461 (local address 127.2.180.254:46055)
0808 15:03:44.741836 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:44.741842 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:44.742076 (+ 234us) server_negotiation.cc:407] Connection header received
0808 15:03:44.742204 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:44.742207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:44.742272 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:44.742350 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:44.743022 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.743738 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.744867 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.745282 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.745375 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:44.749618 (+ 4243us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.195:36461: BlockingRecv error: recv got EOF from 127.2.180.195:36461 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 08 15:03:44 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:44 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.193@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:44.977824 16582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:44.969237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.193:57133)
0808 15:03:44.969739 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:44.969756 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:44.969889 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:44.970266 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:44.970271 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:44.970776 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:44.970973 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:44.970984 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.971964 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.971968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:44.972881 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:44.972893 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.973520 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.973524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:44.973652 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:44.974167 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:44.974183 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:44.977491 (+ 3308us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":239,"threads_started":1}
W20260808 15:03:44.977945 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:44.969257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:57133 (local address 127.2.180.254:46055)
0808 15:03:44.969519 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:44.969525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:44.969816 (+ 291us) server_negotiation.cc:407] Connection header received
0808 15:03:44.970030 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:44.970035 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:44.970129 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:44.970228 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:44.971108 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.971851 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.972987 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:44.973419 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:44.973504 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:44.977682 (+ 4178us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.193:57133: BlockingRecv error: recv got EOF from 127.2.180.193:57133 (error 108)
Metrics: {"server-negotiator.queue_time_us":140}
Aug 08 15:03:45 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 15:03:45 dist-test-slave-jlzn krb5kdc[14951](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.2.180.194@KRBTEST.COM for oryx/127.2.180.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 15:03:45.360073 16583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:45.347479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46055 (local address 127.2.180.194:45937)
0808 15:03:45.347961 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:45.347977 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:45.348109 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:45.348471 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:45.348475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:45.348869 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:45.349060 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:45.349070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.350030 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.350034 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:45.350948 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:45.350960 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.351673 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.351679 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:45.351887 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:45.356552 (+ 4665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:45.356570 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:45.359729 (+ 3159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.2.180.254:46055: Server oryx/127.2.180.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":230,"threads_started":1}
W20260808 15:03:45.360245 16508 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:45.347540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:45937 (local address 127.2.180.254:46055)
0808 15:03:45.347721 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:45.347728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:45.348053 (+ 325us) server_negotiation.cc:407] Connection header received
0808 15:03:45.348224 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:45.348229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:45.348322 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:45.348402 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:45.349208 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.349889 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.351089 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.351530 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.351637 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:45.359979 (+ 8342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.2.180.194:45937: BlockingRecv error: recv got EOF from 127.2.180.194:45937 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:03:45.557199 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 15860
I20260808 15:03:45.568995 16587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:45.558149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.2.180.193:33981)
0808 15:03:45.558748 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:45.558755 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:45.558781 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:03:45.558845 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:45.558850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:45.559083 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:45.559242 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:45.560744 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.561812 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.562921 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.563465 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.563582 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:45.565846 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:45.565854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:45.565867 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:45.565904 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:45.568435 (+ 2531us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.180.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 15:03:45.568722 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:42948: Request ticket server kudu/127.2.180.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":284,"threads_started":1}
W20260808 15:03:45.569037 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 15860: Not authorized: Client connection negotiation failed: client connection to 127.2.180.193:33981: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.180.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 15:03:45.569106 16587 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:42948: Request ticket server kudu/127.2.180.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 15:03:45.569231 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 15860
I20260808 15:03:45.592391 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 16100
I20260808 15:03:45.604460 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:45.593411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60284 (local address 127.2.180.194:40587)
0808 15:03:45.594059 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:45.594065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:45.594095 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:03:45.594157 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:45.594163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:45.594396 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:45.594555 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:45.596157 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.597228 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.598389 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.598951 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.599052 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:45.601501 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:45.601509 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:45.601522 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:45.601566 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:45.603933 (+ 2367us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.180.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 15:03:45.604197 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60284: Request ticket server kudu/127.2.180.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":314,"threads_started":1}
W20260808 15:03:45.604558 16588 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60284: Request ticket server kudu/127.2.180.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 15:03:45.604573 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 16100: Not authorized: Client connection negotiation failed: client connection to 127.2.180.194:40587: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.180.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 15:03:45.604724 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 16100
I20260808 15:03:45.625468 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 16298
I20260808 15:03:45.637377 16589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:45.626607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39258 (local address 127.2.180.195:43173)
0808 15:03:45.627203 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:45.627212 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:45.627243 (+ 31us) server_negotiation.cc:407] Connection header received
0808 15:03:45.627329 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:45.627338 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:45.627642 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:45.627887 (+ 245us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:45.629196 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.630209 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.631334 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:45.632040 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:45.632173 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:45.634491 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:45.634499 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:45.634512 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:45.634558 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:45.636848 (+ 2290us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.180.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 15:03:45.637112 (+ 264us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39258: Request ticket server kudu/127.2.180.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":262,"threads_started":1}
W20260808 15:03:45.637442 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 16298: Not authorized: Client connection negotiation failed: client connection to 127.2.180.195:43173: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.2.180.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 15:03:45.637475 16589 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39258: Request ticket server kudu/127.2.180.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 15:03:45.637621 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 16298
I20260808 15:03:45.664942 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 15751
I20260808 15:03:45.666088 15849 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:45.779906 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 15751
2026-08-08T15:03:45Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21094 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T15:03:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:45Z Disabled control of system clock
I20260808 15:03:45.861490 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40613
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 15:03:46.178514 16596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:46.178781 16596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:46.178900 16596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:46.185837 16596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:46.185940 16596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:46.186002 16596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:46.186085 16596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:46.186144 16596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:46.194895 16596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:46.196110 16596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:46.197778 16596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:46.208854 16604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:46.208832 16602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:46.209307 16601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:46.209635 16596 server_base.cc:1034] running on GCE node
I20260808 15:03:46.210345 16596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:46.211551 16596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:46.212740 16596 hybrid_clock.cc:648] HybridClock initialized: now 1786201426212705 us; error 26 us; skew 500 ppm
I20260808 15:03:46.213063 16596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:46.215058 16596 webserver.cc:460] Webserver started at http://127.2.180.254:37553/ using document root <none> and password file <none>
I20260808 15:03:46.215685 16596 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:46.215785 16596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:46.216107 16596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:46.218835 16596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "922d4493728b45cba1cab51c4bf92fdf"
format_stamp: "Formatted at 2026-08-08 15:03:46 on dist-test-slave-jlzn"
server_key: "85c00a2bf93501b574bf70b2e57757a8"
server_key_iv: "bb9fc9e1b10f77b88dbc70b7473795c5"
server_key_version: "encryptionkey@0"
I20260808 15:03:46.219563 16596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "922d4493728b45cba1cab51c4bf92fdf"
format_stamp: "Formatted at 2026-08-08 15:03:46 on dist-test-slave-jlzn"
server_key: "85c00a2bf93501b574bf70b2e57757a8"
server_key_iv: "bb9fc9e1b10f77b88dbc70b7473795c5"
server_key_version: "encryptionkey@0"
I20260808 15:03:46.224267 16596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 15:03:46.227378 16614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:46.228694 16596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:46.228852 16596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "922d4493728b45cba1cab51c4bf92fdf"
format_stamp: "Formatted at 2026-08-08 15:03:46 on dist-test-slave-jlzn"
server_key: "85c00a2bf93501b574bf70b2e57757a8"
server_key_iv: "bb9fc9e1b10f77b88dbc70b7473795c5"
server_key_version: "encryptionkey@0"
I20260808 15:03:46.229039 16596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:46.258119 16596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:46.258884 16596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:46.259133 16596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:46.275592 16596 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:40613
I20260808 15:03:46.275619 16685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:40613 every 8 connection(s)
I20260808 15:03:46.276793 16596 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:46.279404 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 16596
I20260808 15:03:46.279584 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:46.280290 2771 external_mini_cluster.cc:1428] Setting key afea2001d31f2b9f5e955a98cf5d7d82
I20260808 15:03:46.280387 16687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:46.294607 16693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:46.282191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.2.180.254:40613)
0808 15:03:46.283244 (+ 1053us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:46.283258 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:46.283308 (+ 50us) server_negotiation.cc:407] Connection header received
0808 15:03:46.285162 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:46.285189 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:46.285763 (+ 574us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:46.286163 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:46.286690 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:46.288683 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:46.289900 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:46.290874 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:46.291060 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:46.292092 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:46.292102 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:46.292118 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:46.292796 (+ 678us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:46.292889 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:46.292914 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:46.293022 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:46.293157 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 15:03:46.293436 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":328,"threads_started":1}
I20260808 15:03:46.294914 16687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf: Bootstrap starting.
I20260808 15:03:46.297861 16687 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:46.298768 16687 log.cc:824] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:46.301226 16687 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf: No bootstrap required, opened a new log
I20260808 15:03:46.308161 16687 raft_consensus.cc:348] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d4493728b45cba1cab51c4bf92fdf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40613 } }
I20260808 15:03:46.308527 16687 raft_consensus.cc:374] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:46.308609 16687 raft_consensus.cc:729] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922d4493728b45cba1cab51c4bf92fdf, State: Initialized, Role: FOLLOWER
I20260808 15:03:46.309111 16687 consensus_queue.cc:260] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d4493728b45cba1cab51c4bf92fdf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40613 } }
I20260808 15:03:46.309314 16687 raft_consensus.cc:388] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:46.309407 16687 raft_consensus.cc:482] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:46.309525 16687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:46.311443 16687 raft_consensus.cc:504] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d4493728b45cba1cab51c4bf92fdf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40613 } }
I20260808 15:03:46.311928 16687 leader_election.cc:302] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 922d4493728b45cba1cab51c4bf92fdf; no voters:
I20260808 15:03:46.312418 16687 leader_election.cc:288] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:46.312532 16695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:46.313314 16695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [term 1 LEADER]: Becoming Leader. State: Replica: 922d4493728b45cba1cab51c4bf92fdf, State: Running, Role: LEADER
I20260808 15:03:46.313889 16695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d4493728b45cba1cab51c4bf92fdf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40613 } }
I20260808 15:03:46.314324 16687 sys_catalog.cc:564] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:46.316520 16696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "922d4493728b45cba1cab51c4bf92fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d4493728b45cba1cab51c4bf92fdf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40613 } } }
I20260808 15:03:46.316488 16698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 922d4493728b45cba1cab51c4bf92fdf. Latest consensus state: current_term: 1 leader_uuid: "922d4493728b45cba1cab51c4bf92fdf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d4493728b45cba1cab51c4bf92fdf" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40613 } } }
I20260808 15:03:46.316905 16696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:46.316934 16698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:46.318372 16703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:46.322914 16703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:46.331880 16703 catalog_manager.cc:1347] Generated new cluster ID: f46d066b02164edbb8d9cd4a311b1444
I20260808 15:03:46.331990 16703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:46.359310 16703 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:46.360481 16703 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:46.371152 16703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 922d4493728b45cba1cab51c4bf92fdf: Generated new TSK 0
I20260808 15:03:46.371868 16703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:03:46.380242 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40613
--builtin_ntp_servers=127.2.180.212:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 15:03:46.719934 16720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:46.720187 16720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:46.720268 16720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:46.727409 16720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:46.727499 16720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:46.727622 16720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:46.736526 16720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:46.737665 16720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:46.739437 16720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:46.751974 16729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:46.751947 16732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:46.752004 16734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:46.753028 16720 server_base.cc:1034] running on GCE node
I20260808 15:03:46.753618 16720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:46.754447 16720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:46.755646 16720 hybrid_clock.cc:648] HybridClock initialized: now 1786201426755602 us; error 32 us; skew 500 ppm
I20260808 15:03:46.755978 16720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:46.758147 16720 webserver.cc:460] Webserver started at http://127.2.180.193:42323/ using document root <none> and password file <none>
I20260808 15:03:46.758812 16720 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:46.758910 16720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:46.759222 16720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:46.762036 16720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "554c24d09ddc4a84b539b07b35889ac8"
format_stamp: "Formatted at 2026-08-08 15:03:46 on dist-test-slave-jlzn"
server_key: "69cdf390900bb4e4d1b5684e057cbb76"
server_key_iv: "439ffe2c9d57efb8d5b44ef9f8e480a9"
server_key_version: "encryptionkey@0"
I20260808 15:03:46.762916 16720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "554c24d09ddc4a84b539b07b35889ac8"
format_stamp: "Formatted at 2026-08-08 15:03:46 on dist-test-slave-jlzn"
server_key: "69cdf390900bb4e4d1b5684e057cbb76"
server_key_iv: "439ffe2c9d57efb8d5b44ef9f8e480a9"
server_key_version: "encryptionkey@0"
I20260808 15:03:46.767853 16720 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 15:03:46.771131 16741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:46.772409 16720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 15:03:46.772586 16720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "554c24d09ddc4a84b539b07b35889ac8"
format_stamp: "Formatted at 2026-08-08 15:03:46 on dist-test-slave-jlzn"
server_key: "69cdf390900bb4e4d1b5684e057cbb76"
server_key_iv: "439ffe2c9d57efb8d5b44ef9f8e480a9"
server_key_version: "encryptionkey@0"
I20260808 15:03:46.772778 16720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:46.818475 16720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:46.819324 16720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:46.819604 16720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:46.820554 16720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:46.821964 16720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:46.822059 16720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:46.822168 16720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:46.822242 16720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:46.851856 16720 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:40357
I20260808 15:03:46.851917 16920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:40357 every 8 connection(s)
I20260808 15:03:46.853065 16720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:46.859920 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 16720
I20260808 15:03:46.860105 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:03:46.860754 2771 external_mini_cluster.cc:1428] Setting key 43e7d9baba219ecefb9f42642f56915c
I20260808 15:03:46.863564 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40613
--builtin_ntp_servers=127.2.180.212:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 15:03:46.865801 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:46.855361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:34429 (local address 127.2.180.254:40613)
0808 15:03:46.855798 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:46.855828 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:46.856982 (+ 1154us) server_negotiation.cc:407] Connection header received
0808 15:03:46.858617 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:46.858622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:46.858691 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:46.858786 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:46.859894 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:46.860651 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:46.862514 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:46.862967 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:46.863064 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:46.864638 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:46.864645 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:46.864650 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:46.864674 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:46.864725 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:46.864742 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:46.864836 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:46.865129 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 15:03:46.865284 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":196,"threads_started":1}
I20260808 15:03:46.866766 16928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:46.855995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40613 (local address 127.2.180.193:34429)
0808 15:03:46.856831 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:46.856878 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:46.858389 (+ 1511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:46.858959 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:46.858967 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:46.859044 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:46.859713 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:46.859739 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:46.860811 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:46.860817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:46.862370 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:46.862386 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:46.863222 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:46.863231 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:46.863514 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:46.864257 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:03:46.864260 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:46.864307 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:03:46.864398 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:46.864858 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:46.864869 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:46.864877 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:03:46.865055 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 15:03:46.865434 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":261,"threads_started":1}
I20260808 15:03:46.869181 16923 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40613
I20260808 15:03:46.869603 16923 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:46.870486 16923 heartbeater.cc:511] Master 127.2.180.254:40613 requested a full tablet report, sending...
I20260808 15:03:46.873037 16638 ts_manager.cc:194] Registered new tserver with Master: 554c24d09ddc4a84b539b07b35889ac8 (127.2.180.193:40357)
I20260808 15:03:46.874802 16638 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.193:34429
W20260808 15:03:47.177146 16929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:47.177390 16929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:47.177505 16929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:47.184415 16929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:47.184497 16929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:47.184643 16929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:47.193158 16929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:47.194267 16929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:47.195924 16929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:47.208043 16940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:47.208117 16947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:47.208077 16943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:47.208659 16929 server_base.cc:1034] running on GCE node
I20260808 15:03:47.209281 16929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:47.210175 16929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:47.211366 16929 hybrid_clock.cc:648] HybridClock initialized: now 1786201427211316 us; error 41 us; skew 500 ppm
I20260808 15:03:47.211663 16929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:47.213759 16929 webserver.cc:460] Webserver started at http://127.2.180.194:36937/ using document root <none> and password file <none>
I20260808 15:03:47.214457 16929 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:47.214556 16929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:47.214860 16929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:47.217691 16929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "2b13180b0dc749bd9537f231f0e99d24"
format_stamp: "Formatted at 2026-08-08 15:03:47 on dist-test-slave-jlzn"
server_key: "f4af70e9460cf83a5de26eb950955e80"
server_key_iv: "d5d2bc88b1468de68b53e071b77e2091"
server_key_version: "encryptionkey@0"
I20260808 15:03:47.218439 16929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "2b13180b0dc749bd9537f231f0e99d24"
format_stamp: "Formatted at 2026-08-08 15:03:47 on dist-test-slave-jlzn"
server_key: "f4af70e9460cf83a5de26eb950955e80"
server_key_iv: "d5d2bc88b1468de68b53e071b77e2091"
server_key_version: "encryptionkey@0"
I20260808 15:03:47.223099 16929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 15:03:47.226321 16955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:47.227478 16929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:03:47.227617 16929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "2b13180b0dc749bd9537f231f0e99d24"
format_stamp: "Formatted at 2026-08-08 15:03:47 on dist-test-slave-jlzn"
server_key: "f4af70e9460cf83a5de26eb950955e80"
server_key_iv: "d5d2bc88b1468de68b53e071b77e2091"
server_key_version: "encryptionkey@0"
I20260808 15:03:47.227820 16929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:47.251087 16929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:47.251828 16929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:47.252097 16929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:47.252902 16929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:47.254179 16929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:47.254271 16929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:47.254369 16929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:47.254436 16929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:47.283950 16929 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:45491
I20260808 15:03:47.284029 17148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:45491 every 8 connection(s)
I20260808 15:03:47.285123 16929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:47.287883 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 16929
I20260808 15:03:47.288041 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:03:47.288574 2771 external_mini_cluster.cc:1428] Setting key de855ac36c26d21077c844937abf74aa
I20260808 15:03:47.291590 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40613
--builtin_ntp_servers=127.2.180.212:41879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 15:03:47.297566 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:47.287641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:50673 (local address 127.2.180.254:40613)
0808 15:03:47.287883 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:47.287890 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:47.289426 (+ 1536us) server_negotiation.cc:407] Connection header received
0808 15:03:47.291415 (+ 1989us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:47.291420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:47.291510 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:47.291593 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:47.292859 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.293580 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.295047 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.295429 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.295534 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:47.296824 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:47.296830 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:47.296833 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:47.296850 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:47.296880 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:47.296891 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:47.296963 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:47.297187 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 15:03:47.297279 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 15:03:47.298678 17151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:47.288221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40613 (local address 127.2.180.194:50673)
0808 15:03:47.289267 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:47.289313 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:47.291175 (+ 1862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:47.292085 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:47.292093 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:47.292182 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:47.292707 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:47.292723 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.293739 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.293744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:47.294913 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:47.294925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.295627 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.295635 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:47.295916 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:47.296529 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:03:47.296532 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:47.296567 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:03:47.296638 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:47.296974 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:47.296985 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:47.296993 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:03:47.297137 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 15:03:47.297486 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":294,"threads_started":1}
I20260808 15:03:47.299852 17149 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40613
I20260808 15:03:47.300151 17149 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:47.300951 17149 heartbeater.cc:511] Master 127.2.180.254:40613 requested a full tablet report, sending...
I20260808 15:03:47.302429 16638 ts_manager.cc:194] Registered new tserver with Master: 2b13180b0dc749bd9537f231f0e99d24 (127.2.180.194:45491)
I20260808 15:03:47.303277 16638 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.194:50673
W20260808 15:03:47.608379 17153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:47.608696 17153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:47.608829 17153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:47.615916 17153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:47.616014 17153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:47.616142 17153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:47.624784 17153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:40613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:47.625962 17153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:47.627665 17153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:47.639918 17171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:47.639963 17160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:47.639989 17165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:47.640916 17153 server_base.cc:1034] running on GCE node
I20260808 15:03:47.641450 17153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:47.642261 17153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:47.643460 17153 hybrid_clock.cc:648] HybridClock initialized: now 1786201427643403 us; error 47 us; skew 500 ppm
I20260808 15:03:47.643750 17153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:47.645797 17153 webserver.cc:460] Webserver started at http://127.2.180.195:38211/ using document root <none> and password file <none>
I20260808 15:03:47.646440 17153 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:47.646541 17153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:47.646853 17153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:47.649613 17153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "0c1ae8c7f05e455dacb2031beb4bbf1a"
format_stamp: "Formatted at 2026-08-08 15:03:47 on dist-test-slave-jlzn"
server_key: "4699a2fa77a6f20326db43deb377db51"
server_key_iv: "238886efb7455d9dce3fb61794d6298e"
server_key_version: "encryptionkey@0"
I20260808 15:03:47.650388 17153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "0c1ae8c7f05e455dacb2031beb4bbf1a"
format_stamp: "Formatted at 2026-08-08 15:03:47 on dist-test-slave-jlzn"
server_key: "4699a2fa77a6f20326db43deb377db51"
server_key_iv: "238886efb7455d9dce3fb61794d6298e"
server_key_version: "encryptionkey@0"
I20260808 15:03:47.655483 17153 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260808 15:03:47.658715 17177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:47.659968 17153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:47.660137 17153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "0c1ae8c7f05e455dacb2031beb4bbf1a"
format_stamp: "Formatted at 2026-08-08 15:03:47 on dist-test-slave-jlzn"
server_key: "4699a2fa77a6f20326db43deb377db51"
server_key_iv: "238886efb7455d9dce3fb61794d6298e"
server_key_version: "encryptionkey@0"
I20260808 15:03:47.660326 17153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:47.687098 17153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:47.687911 17153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:47.688186 17153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:47.689069 17153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:47.690408 17153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:47.690515 17153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:47.690621 17153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:47.690696 17153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:47.720212 17153 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:40315
I20260808 15:03:47.720276 17352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:40315 every 8 connection(s)
I20260808 15:03:47.721392 17153 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:47.726533 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 17153
I20260808 15:03:47.726668 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:03:47.727157 2771 external_mini_cluster.cc:1428] Setting key 6cb388d05d8cd8290cf169f4995df17b
I20260808 15:03:47.733485 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:47.723673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:40999 (local address 127.2.180.254:40613)
0808 15:03:47.723885 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:47.723890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:47.725278 (+ 1388us) server_negotiation.cc:407] Connection header received
0808 15:03:47.727077 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:47.727083 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:47.727171 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:47.727283 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:47.728529 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.729488 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.731092 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.731487 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.731625 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:47.732838 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:47.732843 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:47.732846 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:47.732863 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:47.732894 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:47.732906 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:47.732984 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:47.733141 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 15:03:47.733233 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:03:47.734349 17357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:47.724216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40613 (local address 127.2.180.195:40999)
0808 15:03:47.725123 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:47.725166 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:47.726826 (+ 1660us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:47.727458 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:47.727465 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:47.727575 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:47.728247 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:47.728269 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.729664 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.729668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:47.730969 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:47.730982 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.731677 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.731682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:47.731907 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:47.732525 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:03:47.732529 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:47.732573 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:03:47.732661 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:47.732977 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:47.732983 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:47.732988 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:03:47.733119 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 15:03:47.733404 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":280,"threads_started":1}
I20260808 15:03:47.735553 17353 heartbeater.cc:347] Connected to a master server at 127.2.180.254:40613
I20260808 15:03:47.735790 17353 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:47.736491 17353 heartbeater.cc:511] Master 127.2.180.254:40613 requested a full tablet report, sending...
I20260808 15:03:47.737740 16638 ts_manager.cc:194] Registered new tserver with Master: 0c1ae8c7f05e455dacb2031beb4bbf1a (127.2.180.195:40315)
I20260808 15:03:47.738536 16638 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.195:40999
I20260808 15:03:47.742051 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 15:03:47.747869 16927 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:47.746418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40664 (local address 127.2.180.254:40613)
0808 15:03:47.746552 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:47.746556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:47.747025 (+ 469us) server_negotiation.cc:407] Connection header received
0808 15:03:47.747128 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:47.747131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:47.747197 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:47.747270 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:47.747633 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40664: BlockingRecv error: recv got EOF from 127.0.0.1:40664 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:03:47.748386 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 16720
I20260808 15:03:47.755445 17372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:47.749413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48812 (local address 127.2.180.193:40357)
0808 15:03:47.749920 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:47.749926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:47.749948 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:03:47.750002 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:47.750008 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:47.750171 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:47.750286 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:47.750554 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.751475 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.752683 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.753198 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.753289 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:47.754193 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:47.754202 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:47.754212 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:47.754742 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:47.754793 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:47.754812 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:47.754901 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:47.754998 (+ 97us) server_negotiation.cc:299] Negotiation successful
0808 15:03:47.755113 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":271,"threads_started":1}
I20260808 15:03:47.756542 16912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:47.836691 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 16720
I20260808 15:03:47.856329 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 16929
I20260808 15:03:47.863574 17375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:47.857158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.2.180.194:45491)
0808 15:03:47.857632 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:47.857640 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:47.857667 (+ 27us) server_negotiation.cc:407] Connection header received
0808 15:03:47.857726 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:47.857732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:47.857940 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:47.858065 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:47.858407 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.859396 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.860629 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.861169 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.861265 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:47.862148 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:47.862156 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:47.862169 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:47.862755 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:47.862803 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:47.862823 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:47.862905 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:47.862997 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 15:03:47.863130 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":230,"threads_started":1}
I20260808 15:03:47.864816 17142 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:47.943403 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 16929
I20260808 15:03:47.964927 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 17153
I20260808 15:03:47.972179 17380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:47.965773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58852 (local address 127.2.180.195:40315)
0808 15:03:47.966352 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:47.966359 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:47.966392 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:03:47.966451 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:47.966457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:47.966666 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:47.966784 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:47.967135 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.968195 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.969326 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:47.969859 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:47.969950 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:47.970814 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:47.970822 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:47.970835 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:47.971425 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:47.971471 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:47.971492 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:47.971587 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:47.971719 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 15:03:47.971868 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":271,"threads_started":1}
I20260808 15:03:47.973384 17347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:48.052060 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 17153
I20260808 15:03:48.076541 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 16596
I20260808 15:03:48.077862 16678 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:48.177822 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 16596
2026-08-08T15:03:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2393 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T15:03:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:48Z Disabled control of system clock
I20260808 15:03:48.254359 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:32899
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:32899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 15:03:48.565586 17389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:48.565845 17389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:48.565932 17389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:48.572841 17389 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:48.572940 17389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:48.572983 17389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:48.573035 17389 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:48.573078 17389 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:48.582032 17389 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:32899
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:32899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:48.583107 17389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:48.584797 17389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:48.595682 17401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:48.595755 17406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:48.595716 17402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:48.596437 17389 server_base.cc:1034] running on GCE node
I20260808 15:03:48.597157 17389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:48.598399 17389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:48.599606 17389 hybrid_clock.cc:648] HybridClock initialized: now 1786201428599556 us; error 40 us; skew 500 ppm
I20260808 15:03:48.599956 17389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:48.602084 17389 webserver.cc:460] Webserver started at http://127.2.180.254:45579/ using document root <none> and password file <none>
I20260808 15:03:48.602715 17389 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:48.602816 17389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:48.603118 17389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:48.605845 17389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "1401d81f957645e48680e8d850444af0"
format_stamp: "Formatted at 2026-08-08 15:03:48 on dist-test-slave-jlzn"
server_key: "6f3f3d1f7cfe1dc51b4135e9fc78341b"
server_key_iv: "2abfa748bf19693ea4455914f879925e"
server_key_version: "encryptionkey@0"
I20260808 15:03:48.606611 17389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "1401d81f957645e48680e8d850444af0"
format_stamp: "Formatted at 2026-08-08 15:03:48 on dist-test-slave-jlzn"
server_key: "6f3f3d1f7cfe1dc51b4135e9fc78341b"
server_key_iv: "2abfa748bf19693ea4455914f879925e"
server_key_version: "encryptionkey@0"
I20260808 15:03:48.611296 17389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260808 15:03:48.614499 17420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:48.615700 17389 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:03:48.615881 17389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "1401d81f957645e48680e8d850444af0"
format_stamp: "Formatted at 2026-08-08 15:03:48 on dist-test-slave-jlzn"
server_key: "6f3f3d1f7cfe1dc51b4135e9fc78341b"
server_key_iv: "2abfa748bf19693ea4455914f879925e"
server_key_version: "encryptionkey@0"
I20260808 15:03:48.616073 17389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:48.621234 17389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:48.621930 17389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:48.622185 17389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:48.638314 17389 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:32899
I20260808 15:03:48.638351 17493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:32899 every 8 connection(s)
I20260808 15:03:48.639448 17389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:48.641841 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 17389
I20260808 15:03:48.642059 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:48.642858 2771 external_mini_cluster.cc:1428] Setting key 4515173556d437ef316b1fc3d6521e31
I20260808 15:03:48.643260 17495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:48.650789 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:48.644797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49414 (local address 127.2.180.254:32899)
0808 15:03:48.645722 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:48.645731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:48.645773 (+ 42us) server_negotiation.cc:407] Connection header received
0808 15:03:48.647517 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:48.647536 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:48.647951 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:48.648272 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:48.649257 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:48.649263 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:48.649274 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:48.649288 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:48.649367 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:48.649383 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:48.649591 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:48.649748 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 15:03:48.649997 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":345,"threads_started":1}
I20260808 15:03:48.655709 17495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0: Bootstrap starting.
I20260808 15:03:48.658612 17495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:48.659543 17495 log.cc:824] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:48.661852 17495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0: No bootstrap required, opened a new log
I20260808 15:03:48.668704 17495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1401d81f957645e48680e8d850444af0" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32899 } }
I20260808 15:03:48.669008 17495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:48.669111 17495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1401d81f957645e48680e8d850444af0, State: Initialized, Role: FOLLOWER
I20260808 15:03:48.669719 17495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1401d81f957645e48680e8d850444af0" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32899 } }
I20260808 15:03:48.669953 17495 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:48.670077 17495 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:48.670238 17495 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:48.672251 17495 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1401d81f957645e48680e8d850444af0" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32899 } }
I20260808 15:03:48.672683 17495 leader_election.cc:302] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1401d81f957645e48680e8d850444af0; no voters:
I20260808 15:03:48.673225 17495 leader_election.cc:288] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:48.673316 17501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:48.674213 17501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [term 1 LEADER]: Becoming Leader. State: Replica: 1401d81f957645e48680e8d850444af0, State: Running, Role: LEADER
I20260808 15:03:48.674763 17501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1401d81f957645e48680e8d850444af0" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32899 } }
I20260808 15:03:48.675180 17495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:48.677361 17503 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1401d81f957645e48680e8d850444af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1401d81f957645e48680e8d850444af0" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32899 } } }
I20260808 15:03:48.677296 17504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1401d81f957645e48680e8d850444af0. Latest consensus state: current_term: 1 leader_uuid: "1401d81f957645e48680e8d850444af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1401d81f957645e48680e8d850444af0" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32899 } } }
I20260808 15:03:48.678126 17504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:48.678174 17503 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:48.679704 17517 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:48.683558 17517 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:48.691542 17517 catalog_manager.cc:1347] Generated new cluster ID: 5bd4daff583645bfb8678a3b062b2fcf
I20260808 15:03:48.691628 17517 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:48.702764 17517 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:48.703107 17517 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:48.708838 17517 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1401d81f957645e48680e8d850444af0: Generated new TSK 0
I20260808 15:03:48.709470 17517 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:03:48.721365 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32899
--builtin_ntp_servers=127.2.180.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 15:03:49.066174 17533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:49.066435 17533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:49.066570 17533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:49.073467 17533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:49.073565 17533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:49.073709 17533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:49.082459 17533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:49.083551 17533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:49.085441 17533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:49.097947 17546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:49.098017 17542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:49.097982 17543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:49.098876 17533 server_base.cc:1034] running on GCE node
I20260808 15:03:49.099413 17533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:49.100303 17533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:49.101495 17533 hybrid_clock.cc:648] HybridClock initialized: now 1786201429101453 us; error 31 us; skew 500 ppm
I20260808 15:03:49.101776 17533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:49.103854 17533 webserver.cc:460] Webserver started at http://127.2.180.193:36637/ using document root <none> and password file <none>
I20260808 15:03:49.104478 17533 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:49.104573 17533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:49.104884 17533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:49.109624 17533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "b754e87d6ea54a99bb86a446e3559768"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "898f247cd59be90eac5491712f11c0bd"
server_key_iv: "fb52802effe640659b81c04551d0aa74"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.110370 17533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "b754e87d6ea54a99bb86a446e3559768"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "898f247cd59be90eac5491712f11c0bd"
server_key_iv: "fb52802effe640659b81c04551d0aa74"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.115151 17533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:03:49.118363 17555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:49.119561 17533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260808 15:03:49.119712 17533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "b754e87d6ea54a99bb86a446e3559768"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "898f247cd59be90eac5491712f11c0bd"
server_key_iv: "fb52802effe640659b81c04551d0aa74"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.119941 17533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:49.125049 17533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:49.125765 17533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:49.126003 17533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:49.126827 17533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:49.128180 17533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:49.128275 17533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:49.128381 17533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:49.128451 17533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:49.157658 17533 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:43911
I20260808 15:03:49.157711 17727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:43911 every 8 connection(s)
I20260808 15:03:49.158870 17533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:49.166241 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:49.161196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:33677 (local address 127.2.180.254:32899)
0808 15:03:49.161641 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:49.161656 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:49.162742 (+ 1086us) server_negotiation.cc:407] Connection header received
0808 15:03:49.164377 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:49.164381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:49.164433 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:49.164528 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:49.165682 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:49.165686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:49.165689 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:49.165705 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:49.165748 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:49.165760 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:49.165849 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:49.165975 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 15:03:49.166022 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":210,"threads_started":1}
I20260808 15:03:49.167058 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:49.161747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:32899 (local address 127.2.180.193:33677)
0808 15:03:49.162600 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:49.162643 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:49.164156 (+ 1513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:49.164712 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:49.164720 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:49.164803 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:49.165392 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:03:49.165394 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:49.165431 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:03:49.165513 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:49.165831 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:49.165836 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:49.165841 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:03:49.166037 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 15:03:49.166218 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":253,"threads_started":1}
I20260808 15:03:49.168411 17729 heartbeater.cc:347] Connected to a master server at 127.2.180.254:32899
I20260808 15:03:49.168684 17729 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:49.168799 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 17533
I20260808 15:03:49.168999 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:03:49.169265 17729 heartbeater.cc:511] Master 127.2.180.254:32899 requested a full tablet report, sending...
I20260808 15:03:49.169631 2771 external_mini_cluster.cc:1428] Setting key a3a50e56ffb1c324867ebb5b053bea97
I20260808 15:03:49.171236 17447 ts_manager.cc:194] Registered new tserver with Master: b754e87d6ea54a99bb86a446e3559768 (127.2.180.193:43911)
I20260808 15:03:49.172399 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32899
--builtin_ntp_servers=127.2.180.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 15:03:49.486794 17737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:49.487041 17737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:49.487171 17737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:49.494280 17737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:49.494377 17737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:49.494524 17737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:49.503122 17737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:49.504294 17737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:49.505966 17737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:49.517875 17747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:49.517912 17753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:49.517944 17748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:49.518802 17737 server_base.cc:1034] running on GCE node
I20260808 15:03:49.519388 17737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:49.520337 17737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:49.521528 17737 hybrid_clock.cc:648] HybridClock initialized: now 1786201429521476 us; error 44 us; skew 500 ppm
I20260808 15:03:49.521816 17737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:49.523917 17737 webserver.cc:460] Webserver started at http://127.2.180.194:41725/ using document root <none> and password file <none>
I20260808 15:03:49.524539 17737 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:49.524638 17737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:49.524936 17737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:49.527529 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "a721b598a8f54df7be0b40fef1c769ab"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "18b6c5e94862aba9238a1268ae501bad"
server_key_iv: "f65c2a10023913fcf7e67d1611531da8"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.528302 17737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "a721b598a8f54df7be0b40fef1c769ab"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "18b6c5e94862aba9238a1268ae501bad"
server_key_iv: "f65c2a10023913fcf7e67d1611531da8"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.533145 17737 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:03:49.536338 17768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:49.537557 17737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:03:49.537693 17737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "a721b598a8f54df7be0b40fef1c769ab"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "18b6c5e94862aba9238a1268ae501bad"
server_key_iv: "f65c2a10023913fcf7e67d1611531da8"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.537868 17737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:49.542891 17737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:49.543566 17737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:49.543774 17737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:49.544646 17737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:49.545889 17737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:49.545975 17737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:49.546063 17737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:49.546118 17737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:49.575021 17737 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:39089
I20260808 15:03:49.575038 17942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:39089 every 8 connection(s)
I20260808 15:03:49.576320 17737 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:49.583570 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:49.578627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:54525 (local address 127.2.180.254:32899)
0808 15:03:49.578840 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:49.578846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:49.580172 (+ 1326us) server_negotiation.cc:407] Connection header received
0808 15:03:49.581757 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:49.581762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:49.581831 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:49.581919 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:49.583040 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:49.583043 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:49.583046 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:49.583059 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:49.583084 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:49.583094 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:49.583179 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:49.583303 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 15:03:49.583351 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:03:49.584393 17947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:49.579108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:32899 (local address 127.2.180.194:54525)
0808 15:03:49.580003 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:49.580064 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:49.581494 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:49.582104 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:49.582112 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:49.582190 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:49.582800 (+ 610us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:03:49.582802 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:49.582837 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:03:49.582915 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:49.583160 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:49.583165 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:49.583170 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:03:49.583359 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 15:03:49.583529 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":270,"threads_started":1}
I20260808 15:03:49.585561 17944 heartbeater.cc:347] Connected to a master server at 127.2.180.254:32899
I20260808 15:03:49.585690 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 17737
I20260808 15:03:49.585852 17944 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:49.585867 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:03:49.586400 2771 external_mini_cluster.cc:1428] Setting key 329cefc36248818309a03842847a3187
I20260808 15:03:49.586512 17944 heartbeater.cc:511] Master 127.2.180.254:32899 requested a full tablet report, sending...
I20260808 15:03:49.587822 17446 ts_manager.cc:194] Registered new tserver with Master: a721b598a8f54df7be0b40fef1c769ab (127.2.180.194:39089)
I20260808 15:03:49.589263 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32899
--builtin_ntp_servers=127.2.180.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 15:03:49.912495 17948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:49.912750 17948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:49.912868 17948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:49.919790 17948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:49.919915 17948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:49.920064 17948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:49.929111 17948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:49.930192 17948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:49.931906 17948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:49.944043 17962 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:49.944089 17958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:49.944113 17955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:49.945096 17948 server_base.cc:1034] running on GCE node
I20260808 15:03:49.945664 17948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:49.946465 17948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:49.947654 17948 hybrid_clock.cc:648] HybridClock initialized: now 1786201429947616 us; error 30 us; skew 500 ppm
I20260808 15:03:49.947953 17948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:03:49.949930 17948 webserver.cc:460] Webserver started at http://127.2.180.195:44699/ using document root <none> and password file <none>
I20260808 15:03:49.950524 17948 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:49.950619 17948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:49.950871 17948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:49.953543 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "f2c156cc49ab4aa4becdec7fa130e33b"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "e5c20ee0fabc3ce11f3079fc5bec3292"
server_key_iv: "0c598ada26da29b355e09b8c0eaa4b62"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.954278 17948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "f2c156cc49ab4aa4becdec7fa130e33b"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "e5c20ee0fabc3ce11f3079fc5bec3292"
server_key_iv: "0c598ada26da29b355e09b8c0eaa4b62"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.958987 17948 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:03:49.965204 17970 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:49.966400 17948 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.003s
I20260808 15:03:49.966535 17948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "f2c156cc49ab4aa4becdec7fa130e33b"
format_stamp: "Formatted at 2026-08-08 15:03:49 on dist-test-slave-jlzn"
server_key: "e5c20ee0fabc3ce11f3079fc5bec3292"
server_key_iv: "0c598ada26da29b355e09b8c0eaa4b62"
server_key_version: "encryptionkey@0"
I20260808 15:03:49.966713 17948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:49.971844 17948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:49.972522 17948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:49.972741 17948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:49.973520 17948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:49.974794 17948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:49.974882 17948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:49.974958 17948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:49.975025 17948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:50.002957 17948 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:33697
I20260808 15:03:50.003014 18132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:33697 every 8 connection(s)
I20260808 15:03:50.004208 17948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:50.011230 17734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:50.006418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:56343 (local address 127.2.180.254:32899)
0808 15:03:50.006624 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:50.006631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:50.007870 (+ 1239us) server_negotiation.cc:407] Connection header received
0808 15:03:50.009446 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:50.009450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:50.009504 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:50.009588 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:50.010683 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:50.010687 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:50.010690 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:50.010704 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:50.010735 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:50.010746 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:50.010841 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:50.010959 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 15:03:50.011005 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:03:50.012104 18135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:50.006892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:32899 (local address 127.2.180.195:56343)
0808 15:03:50.007713 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:50.007753 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:50.009229 (+ 1476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:50.009764 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:50.009772 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:50.009852 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:50.010429 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:03:50.010432 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:50.010465 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:03:50.010541 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:50.010816 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:50.010822 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:50.010826 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:03:50.011016 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 15:03:50.011200 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":252,"threads_started":1}
I20260808 15:03:50.012401 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 17948
I20260808 15:03:50.012593 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:03:50.013118 2771 external_mini_cluster.cc:1428] Setting key cfe824cad09616cb351a53d671c618b8
I20260808 15:03:50.013340 18133 heartbeater.cc:347] Connected to a master server at 127.2.180.254:32899
I20260808 15:03:50.013633 18133 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:50.014218 18133 heartbeater.cc:511] Master 127.2.180.254:32899 requested a full tablet report, sending...
I20260808 15:03:50.015604 17447 ts_manager.cc:194] Registered new tserver with Master: f2c156cc49ab4aa4becdec7fa130e33b (127.2.180.195:33697)
I20260808 15:03:50.027813 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 15:03:50.033254 17734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:03:50.031899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.2.180.254:32899)
0808 15:03:50.032054 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:50.032059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:50.032532 (+ 473us) server_negotiation.cc:407] Connection header received
0808 15:03:50.032656 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:50.032660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:50.032704 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:50.032774 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:50.033016 (+ 242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49418: BlockingRecv error: recv got EOF from 127.0.0.1:49418 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:03:50.033943 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 17533
I20260808 15:03:50.036850 18150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:50.034734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57474 (local address 127.2.180.193:43911)
0808 15:03:50.035168 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:50.035173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:50.035197 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:03:50.035250 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:50.035255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:50.035404 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:50.035514 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:50.036343 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:50.036349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:50.036358 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:50.036370 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:50.036407 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:50.036420 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:50.036519 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:50.036572 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 15:03:50.036645 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":182,"threads_started":1}
I20260808 15:03:50.037922 17718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:50.118268 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 17533
I20260808 15:03:50.139019 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 17737
I20260808 15:03:50.142462 18153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:50.139871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58826 (local address 127.2.180.194:39089)
0808 15:03:50.140380 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:50.140387 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:50.140414 (+ 27us) server_negotiation.cc:407] Connection header received
0808 15:03:50.140478 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:50.140485 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:50.140699 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:50.140852 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:50.141776 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:50.141782 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:50.141793 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:50.141807 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:50.141850 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:50.141868 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:50.141996 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:50.142084 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 15:03:50.142176 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":224,"threads_started":1}
I20260808 15:03:50.143642 17935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:50.222580 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 17737
I20260808 15:03:50.245560 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 17948
I20260808 15:03:50.248981 18155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:50.246406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55722 (local address 127.2.180.195:33697)
0808 15:03:50.247004 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:50.247012 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:50.247043 (+ 31us) server_negotiation.cc:407] Connection header received
0808 15:03:50.247112 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:50.247119 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:50.247328 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:50.247455 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:50.248372 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:50.248380 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:50.248391 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:03:50.248403 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:50.248443 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:03:50.248460 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:50.248597 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:50.248646 (+ 49us) server_negotiation.cc:299] Negotiation successful
0808 15:03:50.248732 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":273,"threads_started":1}
I20260808 15:03:50.250195 18125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:50.329998 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 17948
I20260808 15:03:50.352465 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 17389
I20260808 15:03:50.353611 17488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:50.451540 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 17389
2026-08-08T15:03:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2270 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:03:50 dist-test-slave-jlzn krb5kdc[18161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:03:50 dist-test-slave-jlzn krb5kdc[18161](info): set up 2 sockets
Aug 08 15:03:50 dist-test-slave-jlzn krb5kdc[18161](info): commencing operation
krb5kdc: starting...
W20260808 15:03:51.635010 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:03:51 dist-test-slave-jlzn krb5kdc[18161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:03:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:03:51.851130 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45571
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45571
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:52.202335 18181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:52.202724 18181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:52.202883 18181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:52.210279 18181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:03:52.210392 18181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:52.210474 18181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:52.210546 18181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:03:52.210603 18181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:03:52.220054 18181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45571
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:52.221227 18181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:52.222416 18181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:52.233160 18187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:52.233323 18191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:52.233292 18194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:52.234226 18181 server_base.cc:1034] running on GCE node
I20260808 15:03:52.234939 18181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:52.236212 18181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:52.237406 18181 hybrid_clock.cc:648] HybridClock initialized: now 1786201432237354 us; error 32 us; skew 500 ppm
I20260808 15:03:52.237708 18181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:52 dist-test-slave-jlzn krb5kdc[18161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:52.240550 18181 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:03:52.241932 18181 webserver.cc:460] Webserver started at http://127.2.180.254:37921/ using document root <none> and password file <none>
I20260808 15:03:52.242532 18181 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:52.242656 18181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:52.242938 18181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:52.245687 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "df5ebe701fb54d69b1d9eac601e21117"
format_stamp: "Formatted at 2026-08-08 15:03:52 on dist-test-slave-jlzn"
server_key: "663e786dc1940a57ccc927e7f5c8ec9e"
server_key_iv: "fb167ceb0387808ac6af90732fba5fc5"
server_key_version: "encryptionkey@0"
I20260808 15:03:52.246393 18181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "df5ebe701fb54d69b1d9eac601e21117"
format_stamp: "Formatted at 2026-08-08 15:03:52 on dist-test-slave-jlzn"
server_key: "663e786dc1940a57ccc927e7f5c8ec9e"
server_key_iv: "fb167ceb0387808ac6af90732fba5fc5"
server_key_version: "encryptionkey@0"
I20260808 15:03:52.251035 18181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260808 15:03:52.254127 18203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:52.255270 18181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:03:52.255419 18181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "df5ebe701fb54d69b1d9eac601e21117"
format_stamp: "Formatted at 2026-08-08 15:03:52 on dist-test-slave-jlzn"
server_key: "663e786dc1940a57ccc927e7f5c8ec9e"
server_key_iv: "fb167ceb0387808ac6af90732fba5fc5"
server_key_version: "encryptionkey@0"
I20260808 15:03:52.255587 18181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:52.278770 18181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:52.282573 18181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:52.282824 18181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:52.299383 18181 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:45571
I20260808 15:03:52.299400 18277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:45571 every 8 connection(s)
I20260808 15:03:52.300781 18181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:03:52.304143 18281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:52.308917 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 18181
I20260808 15:03:52.309147 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:03:52.309885 2771 external_mini_cluster.cc:1428] Setting key 4c145247ebbe207de6e30dcddfe2c6b4
I20260808 15:03:52.315259 18281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117: Bootstrap starting.
I20260808 15:03:52.319026 18281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:52.320292 18281 log.cc:824] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:52.323488 18281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117: No bootstrap required, opened a new log
I20260808 15:03:52.332778 18281 raft_consensus.cc:348] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ebe701fb54d69b1d9eac601e21117" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45571 } }
Aug 08 15:03:52 dist-test-slave-jlzn krb5kdc[18161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:52.333093 18281 raft_consensus.cc:374] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:52.333201 18281 raft_consensus.cc:729] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df5ebe701fb54d69b1d9eac601e21117, State: Initialized, Role: FOLLOWER
I20260808 15:03:52.333825 18281 consensus_queue.cc:260] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ebe701fb54d69b1d9eac601e21117" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45571 } }
I20260808 15:03:52.334097 18281 raft_consensus.cc:388] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:03:52.334239 18281 raft_consensus.cc:482] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:03:52.334416 18281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:52.336841 18281 raft_consensus.cc:504] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ebe701fb54d69b1d9eac601e21117" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45571 } }
I20260808 15:03:52.337299 18281 leader_election.cc:302] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df5ebe701fb54d69b1d9eac601e21117; no voters:
I20260808 15:03:52.337932 18281 leader_election.cc:288] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:03:52.338030 18290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:52.340217 18281 sys_catalog.cc:564] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:03:52.340161 18290 raft_consensus.cc:686] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [term 1 LEADER]: Becoming Leader. State: Replica: df5ebe701fb54d69b1d9eac601e21117, State: Running, Role: LEADER
I20260808 15:03:52.340716 18290 consensus_queue.cc:237] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ebe701fb54d69b1d9eac601e21117" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45571 } }
I20260808 15:03:52.342497 18285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:52.316685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38936 (local address 127.2.180.254:45571)
0808 15:03:52.320665 (+ 3980us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:52.320679 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:52.320723 (+ 44us) server_negotiation.cc:407] Connection header received
0808 15:03:52.323079 (+ 2356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:52.323107 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:52.323515 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:52.323931 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:52.325805 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:52.327116 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:52.328898 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:52.329553 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:52.329705 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:52.334243 (+ 4538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:52.334252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:52.334264 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:52.334298 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:52.337265 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:52.338033 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:52.338041 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:52.338049 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:52.338289 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:52.338771 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:52.338777 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:52.338780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:52.339454 (+ 674us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:52.339769 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:52.340540 (+ 771us) server_negotiation.cc:299] Negotiation successful
0808 15:03:52.341071 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3710,"thread_start_us":4578,"threads_started":1}
I20260808 15:03:52.343442 18291 sys_catalog.cc:455] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df5ebe701fb54d69b1d9eac601e21117" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ebe701fb54d69b1d9eac601e21117" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45571 } } }
I20260808 15:03:52.343776 18291 sys_catalog.cc:458] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [sys.catalog]: This master's current role is: LEADER
I20260808 15:03:52.349115 18292 sys_catalog.cc:455] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df5ebe701fb54d69b1d9eac601e21117. Latest consensus state: current_term: 1 leader_uuid: "df5ebe701fb54d69b1d9eac601e21117" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5ebe701fb54d69b1d9eac601e21117" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45571 } } }
I20260808 15:03:52.349462 18292 sys_catalog.cc:458] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117 [sys.catalog]: This master's current role is: LEADER
W20260808 15:03:52.352236 18311 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:03:52.352362 18311 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:03:52.353001 18298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:03:52.355099 18298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:03:52.362839 18298 catalog_manager.cc:1347] Generated new cluster ID: 453b76d4a6754b30b603ccb869e8fc06
I20260808 15:03:52.362936 18298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:03:52.373297 18298 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:03:52.374961 18298 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:03:52.384584 18298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117: Generated new TSK 0
I20260808 15:03:52.385237 18298 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:03:52.445125 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45571
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:52.762347 18318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:52.762614 18318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:52.762732 18318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:52.769676 18318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:52.769778 18318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:52.769928 18318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:03:52.779143 18318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:52.780304 18318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:52.781559 18318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:52.794366 18332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:52.794462 18330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:52.795084 18334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:52.795142 18318 server_base.cc:1034] running on GCE node
I20260808 15:03:52.795729 18318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:52.796614 18318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:52.797820 18318 hybrid_clock.cc:648] HybridClock initialized: now 1786201432797781 us; error 24 us; skew 500 ppm
I20260808 15:03:52.798108 18318 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:52 dist-test-slave-jlzn krb5kdc[18161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:52.801115 18318 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:03:52.802431 18318 webserver.cc:460] Webserver started at http://127.2.180.193:46433/ using document root <none> and password file <none>
I20260808 15:03:52.803107 18318 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:52.803227 18318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:52.803505 18318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:52.806311 18318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "1af3b9b15f594749b8b10e3c789a7dec"
format_stamp: "Formatted at 2026-08-08 15:03:52 on dist-test-slave-jlzn"
server_key: "6bce0b25c509c4ac8c1d737cf0b9f7ff"
server_key_iv: "e365fd28d27e820861ae04405517d943"
server_key_version: "encryptionkey@0"
I20260808 15:03:52.807058 18318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "1af3b9b15f594749b8b10e3c789a7dec"
format_stamp: "Formatted at 2026-08-08 15:03:52 on dist-test-slave-jlzn"
server_key: "6bce0b25c509c4ac8c1d737cf0b9f7ff"
server_key_iv: "e365fd28d27e820861ae04405517d943"
server_key_version: "encryptionkey@0"
I20260808 15:03:52.811834 18318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 15:03:52.815047 18342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:52.816242 18318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:03:52.816385 18318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "1af3b9b15f594749b8b10e3c789a7dec"
format_stamp: "Formatted at 2026-08-08 15:03:52 on dist-test-slave-jlzn"
server_key: "6bce0b25c509c4ac8c1d737cf0b9f7ff"
server_key_iv: "e365fd28d27e820861ae04405517d943"
server_key_version: "encryptionkey@0"
I20260808 15:03:52.816581 18318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:52.847559 18318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:52.851632 18318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:52.851946 18318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:52.852962 18318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:52.854305 18318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:52.854383 18318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:52.854483 18318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:52.854535 18318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:52.887324 18318 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:33439
I20260808 15:03:52.887362 18528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:33439 every 8 connection(s)
I20260808 15:03:52.888609 18318 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:03:52.894285 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 18318
I20260808 15:03:52.894512 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:03:52.895118 2771 external_mini_cluster.cc:1428] Setting key 41e4210fef23ee86a6375956da93ddd5
Aug 08 15:03:52 dist-test-slave-jlzn krb5kdc[18161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201432, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:52.907440 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:52.890882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:34313 (local address 127.2.180.254:45571)
0808 15:03:52.891342 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:52.891347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:52.892272 (+ 925us) server_negotiation.cc:407] Connection header received
0808 15:03:52.894067 (+ 1795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:52.894074 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:52.894184 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:52.894309 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:52.895785 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:52.896804 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:52.898300 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:52.898700 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:52.898808 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:52.902172 (+ 3364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:52.902177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:52.902181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:52.902213 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:52.904601 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:52.905272 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:52.905278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:52.905281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:52.905400 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:52.905807 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:52.905813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:52.905815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:52.906076 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:52.906211 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:52.906869 (+ 658us) server_negotiation.cc:299] Negotiation successful
0808 15:03:52.907087 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":209,"threads_started":1}
I20260808 15:03:52.908339 18532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:52.891436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45571 (local address 127.2.180.193:34313)
0808 15:03:52.892130 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:52.892168 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:52.893805 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:52.894531 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:52.894540 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:52.895040 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:52.895616 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:52.895634 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:52.896967 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:52.896971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:52.898159 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:52.898172 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:52.898918 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:52.898926 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:52.899148 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:52.899742 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:52.899763 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:52.901961 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:52.904731 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:52.904740 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:52.904755 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:52.905160 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:52.905503 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:52.905508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:52.905512 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:52.905699 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:52.906201 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:52.906208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:52.906510 (+ 302us) client_negotiation.cc:769] Sending connection context
0808 15:03:52.906744 (+ 234us) client_negotiation.cc:240] Negotiation successful
0808 15:03:52.907224 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":277,"threads_started":1}
I20260808 15:03:52.909783 18529 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45571
I20260808 15:03:52.910070 18529 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:52.910928 18529 heartbeater.cc:511] Master 127.2.180.254:45571 requested a full tablet report, sending...
I20260808 15:03:52.913214 18221 ts_manager.cc:194] Registered new tserver with Master: 1af3b9b15f594749b8b10e3c789a7dec (127.2.180.193:33439)
I20260808 15:03:52.915036 18221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:34313
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:03:52.946471 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45571
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:53.264068 18538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:53.264334 18538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:53.264424 18538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:53.271288 18538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:53.271386 18538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:53.271517 18538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:03:53.281006 18538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:53.282179 18538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:53.283396 18538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:53.295864 18549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:53.295907 18556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:53.295941 18553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:53.296867 18538 server_base.cc:1034] running on GCE node
I20260808 15:03:53.297472 18538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:53.298373 18538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:53.299564 18538 hybrid_clock.cc:648] HybridClock initialized: now 1786201433299513 us; error 47 us; skew 500 ppm
I20260808 15:03:53.299904 18538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:53 dist-test-slave-jlzn krb5kdc[18161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201433, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:53.303066 18538 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:03:53.304600 18538 webserver.cc:460] Webserver started at http://127.2.180.194:40607/ using document root <none> and password file <none>
I20260808 15:03:53.305264 18538 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:53.305387 18538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:53.305681 18538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:53.308511 18538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "df6d90904a22473c926159db2cee1b24"
format_stamp: "Formatted at 2026-08-08 15:03:53 on dist-test-slave-jlzn"
server_key: "241d0e97693040f96d246f827f040e39"
server_key_iv: "6c5ad93f773325bcb2a1adb50efbb9fe"
server_key_version: "encryptionkey@0"
I20260808 15:03:53.309266 18538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "df6d90904a22473c926159db2cee1b24"
format_stamp: "Formatted at 2026-08-08 15:03:53 on dist-test-slave-jlzn"
server_key: "241d0e97693040f96d246f827f040e39"
server_key_iv: "6c5ad93f773325bcb2a1adb50efbb9fe"
server_key_version: "encryptionkey@0"
I20260808 15:03:53.314283 18538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 15:03:53.317579 18571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:53.318795 18538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 15:03:53.318938 18538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "df6d90904a22473c926159db2cee1b24"
format_stamp: "Formatted at 2026-08-08 15:03:53 on dist-test-slave-jlzn"
server_key: "241d0e97693040f96d246f827f040e39"
server_key_iv: "6c5ad93f773325bcb2a1adb50efbb9fe"
server_key_version: "encryptionkey@0"
I20260808 15:03:53.319110 18538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:53.330498 18538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:53.334118 18538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:53.334370 18538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:53.335201 18538 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:53.336467 18538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:53.336539 18538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:53.336650 18538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:53.336700 18538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:53.365392 18538 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:43283
I20260808 15:03:53.365432 18752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:43283 every 8 connection(s)
I20260808 15:03:53.366683 18538 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:03:53.375581 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 18538
I20260808 15:03:53.375783 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:03:53.376312 2771 external_mini_cluster.cc:1428] Setting key 0e3724bd431a6ad3470e45a8552e2413
Aug 08 15:03:53 dist-test-slave-jlzn krb5kdc[18161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201433, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:53.385601 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.369134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:55213 (local address 127.2.180.254:45571)
0808 15:03:53.369348 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:53.369354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:53.370496 (+ 1142us) server_negotiation.cc:407] Connection header received
0808 15:03:53.372244 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:53.372249 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:53.372317 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:53.372402 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:53.373731 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.374432 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.376229 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.376720 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.376836 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.380240 (+ 3404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:53.380245 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:53.380248 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:53.380284 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:53.382721 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:53.383392 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:53.383396 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:53.383399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:53.383515 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:53.383947 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:53.383951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:53.383953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:53.384225 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:53.384357 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:53.385051 (+ 694us) server_negotiation.cc:299] Negotiation successful
0808 15:03:53.385286 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 15:03:53.386425 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.369669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45571 (local address 127.2.180.194:55213)
0808 15:03:53.370351 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:53.370388 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:53.371999 (+ 1611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:53.372571 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:53.372579 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:53.373059 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:53.373585 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.373601 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.374538 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.374541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.376097 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.376112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.376936 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.376944 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.377161 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.377763 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:53.377785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:53.380027 (+ 2242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:53.382847 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:53.382855 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:53.382870 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:53.383267 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:53.383623 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:53.383628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:53.383632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:53.383836 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:53.384350 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:53.384357 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:53.384674 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 15:03:53.384896 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 15:03:53.385374 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":260,"threads_started":1}
I20260808 15:03:53.387701 18753 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45571
I20260808 15:03:53.388041 18753 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:53.388742 18753 heartbeater.cc:511] Master 127.2.180.254:45571 requested a full tablet report, sending...
I20260808 15:03:53.390071 18221 ts_manager.cc:194] Registered new tserver with Master: df6d90904a22473c926159db2cee1b24 (127.2.180.194:43283)
I20260808 15:03:53.390859 18221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:55213
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:03:53.431046 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45571
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:03:53.746773 18761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:03:53.747056 18761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:03:53.747169 18761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:03:53.754248 18761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:03:53.754346 18761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:03:53.754467 18761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:03:53.763818 18761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45571
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:03:53.764923 18761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:03:53.766147 18761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:03:53.778340 18771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:53.778306 18772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:03:53.778379 18774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:03:53.779367 18761 server_base.cc:1034] running on GCE node
I20260808 15:03:53.779925 18761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:03:53.780737 18761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:03:53.781940 18761 hybrid_clock.cc:648] HybridClock initialized: now 1786201433781886 us; error 43 us; skew 500 ppm
I20260808 15:03:53.782210 18761 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:03:53 dist-test-slave-jlzn krb5kdc[18161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201433, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:03:53.785202 18761 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:03:53.786552 18761 webserver.cc:460] Webserver started at http://127.2.180.195:36223/ using document root <none> and password file <none>
I20260808 15:03:53.787153 18761 fs_manager.cc:359] Metadata directory not provided
I20260808 15:03:53.787261 18761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:03:53.787541 18761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:03:53.790309 18761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "7f4e5215283d4a398bad003c1e63b434"
format_stamp: "Formatted at 2026-08-08 15:03:53 on dist-test-slave-jlzn"
server_key: "d35f99f98c24704237b7a82939876cdf"
server_key_iv: "c2284409f5301bdc5e240d7ebc58e21e"
server_key_version: "encryptionkey@0"
I20260808 15:03:53.791057 18761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "7f4e5215283d4a398bad003c1e63b434"
format_stamp: "Formatted at 2026-08-08 15:03:53 on dist-test-slave-jlzn"
server_key: "d35f99f98c24704237b7a82939876cdf"
server_key_iv: "c2284409f5301bdc5e240d7ebc58e21e"
server_key_version: "encryptionkey@0"
I20260808 15:03:53.795913 18761 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 15:03:53.799062 18784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:53.800280 18761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 15:03:53.800428 18761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "7f4e5215283d4a398bad003c1e63b434"
format_stamp: "Formatted at 2026-08-08 15:03:53 on dist-test-slave-jlzn"
server_key: "d35f99f98c24704237b7a82939876cdf"
server_key_iv: "c2284409f5301bdc5e240d7ebc58e21e"
server_key_version: "encryptionkey@0"
I20260808 15:03:53.800604 18761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:03:53.817827 18761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:03:53.821563 18761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:03:53.821805 18761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:03:53.822705 18761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:03:53.824055 18761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:03:53.824133 18761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:53.824263 18761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:03:53.824316 18761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:03:53.853549 18761 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:39599
I20260808 15:03:53.853590 18956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:39599 every 8 connection(s)
I20260808 15:03:53.854710 18761 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:03:53.863615 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 18761
I20260808 15:03:53.863790 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:03:53.864339 2771 external_mini_cluster.cc:1428] Setting key f975b3d3a60e5a681d9d820313ad46f5
Aug 08 15:03:53 dist-test-slave-jlzn krb5kdc[18161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201433, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:03:53.873447 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.856938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:35539 (local address 127.2.180.254:45571)
0808 15:03:53.857101 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:53.857106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:53.858217 (+ 1111us) server_negotiation.cc:407] Connection header received
0808 15:03:53.860046 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:53.860052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:53.860151 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:53.860243 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:53.861651 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.862425 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.864072 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.864690 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.864821 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.868416 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:53.868421 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:53.868424 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:53.868455 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:53.870635 (+ 2180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:53.871291 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:53.871296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:53.871298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:53.871413 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:53.871854 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:53.871858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:53.871860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:53.872122 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:53.872255 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:53.872904 (+ 649us) server_negotiation.cc:299] Negotiation successful
0808 15:03:53.873116 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:03:53.874269 18962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.857370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45571 (local address 127.2.180.195:35539)
0808 15:03:53.858039 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:53.858084 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:53.859739 (+ 1655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:53.860481 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:53.860489 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:53.860953 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:03:53.861490 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.861507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.862579 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.862583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.863912 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.863927 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.864923 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.864932 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.865202 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.865843 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:03:53.865881 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:03:53.868231 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:03:53.870761 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:53.870769 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:53.870784 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:53.871170 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:53.871526 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:03:53.871531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:03:53.871535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:03:53.871711 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:03:53.872248 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:03:53.872255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:03:53.872543 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 15:03:53.872771 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 15:03:53.873238 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":262,"threads_started":1}
I20260808 15:03:53.875398 18958 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45571
I20260808 15:03:53.875640 18958 heartbeater.cc:464] Registering TS with master...
I20260808 15:03:53.876324 18958 heartbeater.cc:511] Master 127.2.180.254:45571 requested a full tablet report, sending...
I20260808 15:03:53.877591 18221 ts_manager.cc:194] Registered new tserver with Master: 7f4e5215283d4a398bad003c1e63b434 (127.2.180.195:39599)
I20260808 15:03:53.878582 18221 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:35539
I20260808 15:03:53.879468 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:03:53.895985 18531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.884032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38942 (local address 127.2.180.254:45571)
0808 15:03:53.884176 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:53.884180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:53.884660 (+ 480us) server_negotiation.cc:407] Connection header received
0808 15:03:53.884774 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:53.884777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:53.884829 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:53.884902 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:53.886145 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.886808 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.888546 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.888943 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.889034 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.891653 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:53.891658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:53.891661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:53.891692 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:53.894003 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:53.894528 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:53.894531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:53.894533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:53.894676 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:53.894930 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:53.894933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:53.894935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:53.895182 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:53.895358 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:53.895588 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 15:03:53.895733 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:03:53.899490 18220 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:03:53.906087 18220 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:03:53.918220 18529 heartbeater.cc:503] Master 127.2.180.254:45571 was elected leader, sending a full tablet report...
I20260808 15:03:53.939383 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.916772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33898 (local address 127.2.180.194:43283)
0808 15:03:53.932785 (+ 16013us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:53.932793 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:53.932819 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:03:53.932904 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:53.932912 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:53.933110 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:53.933348 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:53.933921 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.935345 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.937181 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.938404 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.938538 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.938726 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:53.938933 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 15:03:53.939105 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15852,"thread_start_us":15758,"threads_started":1}
I20260808 15:03:53.939401 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.916516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:43283 (local address 127.0.0.1:33898)
0808 15:03:53.926016 (+ 9500us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:53.926067 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:53.926229 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:53.933342 (+ 7113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:53.933347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:53.933372 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:53.933709 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.933722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.935471 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.935478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.937032 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.937044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.938597 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.938602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.938728 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.938764 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:03:53.938869 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 15:03:53.939036 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9265,"thread_start_us":13336,"threads_started":1}
I20260808 15:03:53.940543 18982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.917208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:33439 (local address 127.0.0.1:51230)
0808 15:03:53.931660 (+ 14452us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:53.931685 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:53.931836 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:53.932789 (+ 953us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:53.932795 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:53.932815 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:53.933114 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.933122 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.934751 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.934757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.937391 (+ 2634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.937407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.939439 (+ 2032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.939446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.939879 (+ 433us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.939914 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 15:03:53.940010 (+ 96us) client_negotiation.cc:240] Negotiation successful
0808 15:03:53.940131 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14295,"mutex_wait_us":119,"thread_start_us":8425,"threads_started":1}
I20260808 15:03:53.940606 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.917782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51230 (local address 127.2.180.193:33439)
0808 15:03:53.931343 (+ 13561us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:53.931352 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:53.931961 (+ 609us) server_negotiation.cc:407] Connection header received
0808 15:03:53.932061 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:53.932068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:53.932384 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:53.932571 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:53.933282 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.934601 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.937564 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.938959 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.939088 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.939293 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:53.940112 (+ 819us) server_negotiation.cc:299] Negotiation successful
0808 15:03:53.940305 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13401,"thread_start_us":13268,"threads_started":1}
I20260808 15:03:53.941895 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.917322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33282 (local address 127.2.180.195:39599)
0808 15:03:53.931793 (+ 14471us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:53.931821 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:53.931853 (+ 32us) server_negotiation.cc:407] Connection header received
0808 15:03:53.931940 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:53.931947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:53.932186 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:53.933014 (+ 828us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:53.933263 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.934714 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.938823 (+ 4109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.940168 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.940998 (+ 830us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.941207 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:53.941364 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 15:03:53.941515 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14300,"thread_start_us":14917,"threads_started":1}
I20260808 15:03:53.943608 18652 tablet_service.cc:1467] Processing CreateTablet for tablet 766609a06c4242f7b1f3409b63cf0c82 (DEFAULT_TABLE table=test-table [id=55ba4a7c89a44803b2596a118e8a2d2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:53.944298 18419 tablet_service.cc:1467] Processing CreateTablet for tablet 766609a06c4242f7b1f3409b63cf0c82 (DEFAULT_TABLE table=test-table [id=55ba4a7c89a44803b2596a118e8a2d2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:53.945206 18652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 766609a06c4242f7b1f3409b63cf0c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:53.945688 18419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 766609a06c4242f7b1f3409b63cf0c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:53.947938 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:53.916776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:39599 (local address 127.0.0.1:33282)
0808 15:03:53.929674 (+ 12898us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:53.929695 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:53.929790 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:53.932410 (+ 2620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:53.932417 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:53.932447 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:53.932820 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.932835 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.934872 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.934878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.937408 (+ 2530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:53.937418 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:53.940387 (+ 2969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:53.940393 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:53.940593 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:53.940626 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:03:53.941325 (+ 699us) client_negotiation.cc:240] Negotiation successful
0808 15:03:53.941448 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12784,"mutex_wait_us":83,"thread_start_us":14562,"threads_started":1}
I20260808 15:03:53.950654 18851 tablet_service.cc:1467] Processing CreateTablet for tablet 766609a06c4242f7b1f3409b63cf0c82 (DEFAULT_TABLE table=test-table [id=55ba4a7c89a44803b2596a118e8a2d2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:03:53.951707 18851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 766609a06c4242f7b1f3409b63cf0c82. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:03:53.960250 18988 tablet_bootstrap.cc:492] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: Bootstrap starting.
I20260808 15:03:53.960866 18987 tablet_bootstrap.cc:492] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: Bootstrap starting.
I20260808 15:03:53.963532 18989 tablet_bootstrap.cc:492] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Bootstrap starting.
I20260808 15:03:53.963694 18988 tablet_bootstrap.cc:654] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:53.964174 18987 tablet_bootstrap.cc:654] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:53.965008 18988 log.cc:824] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:53.965127 18987 log.cc:824] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:53.966755 18989 tablet_bootstrap.cc:654] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Neither blocks nor log segments found. Creating new log.
I20260808 15:03:53.967602 18989 log.cc:824] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Log is configured to *not* fsync() on all Append() calls
I20260808 15:03:53.967993 18988 tablet_bootstrap.cc:492] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: No bootstrap required, opened a new log
I20260808 15:03:53.968264 18988 ts_tablet_manager.cc:1397] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 15:03:53.968418 18987 tablet_bootstrap.cc:492] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: No bootstrap required, opened a new log
I20260808 15:03:53.968664 18987 ts_tablet_manager.cc:1397] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 15:03:53.970623 18989 tablet_bootstrap.cc:492] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: No bootstrap required, opened a new log
I20260808 15:03:53.970853 18989 ts_tablet_manager.cc:1397] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 15:03:53.977602 18989 raft_consensus.cc:348] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:53.977927 18989 raft_consensus.cc:374] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:53.978026 18989 raft_consensus.cc:729] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f4e5215283d4a398bad003c1e63b434, State: Initialized, Role: FOLLOWER
I20260808 15:03:53.978365 18988 raft_consensus.cc:348] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:53.978369 18987 raft_consensus.cc:348] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:53.978772 18987 raft_consensus.cc:374] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:53.978853 18988 raft_consensus.cc:374] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:03:53.978703 18989 consensus_queue.cc:260] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:53.978911 18987 raft_consensus.cc:729] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df6d90904a22473c926159db2cee1b24, State: Initialized, Role: FOLLOWER
I20260808 15:03:53.979032 18988 raft_consensus.cc:729] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1af3b9b15f594749b8b10e3c789a7dec, State: Initialized, Role: FOLLOWER
I20260808 15:03:53.979514 18987 consensus_queue.cc:260] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:53.979903 18958 heartbeater.cc:503] Master 127.2.180.254:45571 was elected leader, sending a full tablet report...
I20260808 15:03:53.980561 18989 ts_tablet_manager.cc:1428] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260808 15:03:53.980695 18753 heartbeater.cc:503] Master 127.2.180.254:45571 was elected leader, sending a full tablet report...
I20260808 15:03:53.979666 18988 consensus_queue.cc:260] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:53.981261 18987 ts_tablet_manager.cc:1428] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: Time spent starting tablet: real 0.012s user 0.007s sys 0.005s
I20260808 15:03:53.982815 18988 ts_tablet_manager.cc:1428] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260808 15:03:54.025833 18995 raft_consensus.cc:482] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:03:54.026171 18995 raft_consensus.cc:504] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:54.027787 18995 leader_election.cc:288] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f4e5215283d4a398bad003c1e63b434 (127.2.180.195:39599), 1af3b9b15f594749b8b10e3c789a7dec (127.2.180.193:33439)
I20260808 15:03:54.034339 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:54.028223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:37055 (local address 127.2.180.195:39599)
0808 15:03:54.028426 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:54.028432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:54.029219 (+ 787us) server_negotiation.cc:407] Connection header received
0808 15:03:54.029397 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:54.029401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:54.029459 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:54.029540 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:54.030059 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.030789 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.032707 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.033453 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.033616 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:54.033692 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:54.033973 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 15:03:54.034096 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 15:03:54.034400 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:54.028291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:52693 (local address 127.2.180.193:33439)
0808 15:03:54.028481 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:54.028487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:54.029304 (+ 817us) server_negotiation.cc:407] Connection header received
0808 15:03:54.029462 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:54.029468 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:54.029529 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:54.029598 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:03:54.030199 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.030912 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.032800 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.033551 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.033693 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:54.033762 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:54.034042 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 15:03:54.034174 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:03:54.034544 18998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:54.028175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:39599 (local address 127.2.180.194:37055)
0808 15:03:54.029112 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:54.029127 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:54.029264 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:54.029558 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:54.029562 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:54.029585 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:54.029918 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:54.029927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.030913 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.030918 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:54.032549 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:54.032562 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.033679 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.033685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:54.033818 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:54.033844 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 15:03:54.033930 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 15:03:54.034060 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"mutex_wait_us":54,"thread_start_us":513,"threads_started":1}
I20260808 15:03:54.034592 18999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:54.028292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:33439 (local address 127.2.180.194:52693)
0808 15:03:54.029212 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0808 15:03:54.029227 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:03:54.029344 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:03:54.029629 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:03:54.029637 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:03:54.029683 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:03:54.030053 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:54.030062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.031050 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.031053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:54.032646 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:03:54.032666 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.033762 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.033768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:03:54.033907 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:54.033928 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 15:03:54.034003 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 15:03:54.034118 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":24,"thread_start_us":599,"threads_started":1}
I20260808 15:03:54.035411 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "766609a06c4242f7b1f3409b63cf0c82" candidate_uuid: "df6d90904a22473c926159db2cee1b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f4e5215283d4a398bad003c1e63b434" is_pre_election: true
I20260808 15:03:54.035423 18449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "766609a06c4242f7b1f3409b63cf0c82" candidate_uuid: "df6d90904a22473c926159db2cee1b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1af3b9b15f594749b8b10e3c789a7dec" is_pre_election: true
I20260808 15:03:54.035863 18882 raft_consensus.cc:2454] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df6d90904a22473c926159db2cee1b24 in term 0.
I20260808 15:03:54.035902 18449 raft_consensus.cc:2454] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df6d90904a22473c926159db2cee1b24 in term 0.
I20260808 15:03:54.036444 18576 leader_election.cc:302] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f4e5215283d4a398bad003c1e63b434, df6d90904a22473c926159db2cee1b24; no voters:
I20260808 15:03:54.036780 18995 raft_consensus.cc:2781] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:03:54.036896 18995 raft_consensus.cc:482] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:03:54.036966 18995 raft_consensus.cc:3037] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:54.039140 18995 raft_consensus.cc:504] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:54.039642 18995 leader_election.cc:288] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [CANDIDATE]: Term 1 election: Requested vote from peers 7f4e5215283d4a398bad003c1e63b434 (127.2.180.195:39599), 1af3b9b15f594749b8b10e3c789a7dec (127.2.180.193:33439)
I20260808 15:03:54.040022 18882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "766609a06c4242f7b1f3409b63cf0c82" candidate_uuid: "df6d90904a22473c926159db2cee1b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f4e5215283d4a398bad003c1e63b434"
I20260808 15:03:54.040115 18449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "766609a06c4242f7b1f3409b63cf0c82" candidate_uuid: "df6d90904a22473c926159db2cee1b24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1af3b9b15f594749b8b10e3c789a7dec"
I20260808 15:03:54.040220 18882 raft_consensus.cc:3037] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:54.040284 18449 raft_consensus.cc:3037] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:03:54.043241 18449 raft_consensus.cc:2454] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df6d90904a22473c926159db2cee1b24 in term 1.
I20260808 15:03:54.043242 18882 raft_consensus.cc:2454] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df6d90904a22473c926159db2cee1b24 in term 1.
I20260808 15:03:54.043623 18575 leader_election.cc:302] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1af3b9b15f594749b8b10e3c789a7dec, df6d90904a22473c926159db2cee1b24; no voters:
I20260808 15:03:54.043919 18995 raft_consensus.cc:2781] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:03:54.044334 18995 raft_consensus.cc:686] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 1 LEADER]: Becoming Leader. State: Replica: df6d90904a22473c926159db2cee1b24, State: Running, Role: LEADER
I20260808 15:03:54.044795 18995 consensus_queue.cc:237] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } }
I20260808 15:03:54.049072 18220 catalog_manager.cc:5495] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 reported cstate change: term changed from 0 to 1, leader changed from <none> to df6d90904a22473c926159db2cee1b24 (127.2.180.194). New cstate: current_term: 1 leader_uuid: "df6d90904a22473c926159db2cee1b24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df6d90904a22473c926159db2cee1b24" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 43283 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 } health_report { overall_health: UNKNOWN } } }
I20260808 15:03:54.089689 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:54.085353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33914 (local address 127.2.180.194:43283)
0808 15:03:54.085508 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:54.085512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:54.085547 (+ 35us) server_negotiation.cc:407] Connection header received
0808 15:03:54.085622 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:54.085625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:54.085701 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:54.085809 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:03:54.086086 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.086744 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.087965 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.088371 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.088460 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:54.088814 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:03:54.089270 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:03:54.089370 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:54.089423 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 15:03:54.089482 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:03:54.096987 18449 raft_consensus.cc:1261] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 1 FOLLOWER]: Refusing update from remote peer df6d90904a22473c926159db2cee1b24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:54.097000 18882 raft_consensus.cc:1261] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 1 FOLLOWER]: Refusing update from remote peer df6d90904a22473c926159db2cee1b24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:03:54.097788 19001 consensus_queue.cc:1035] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1af3b9b15f594749b8b10e3c789a7dec" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33439 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:03:54.098129 18995 consensus_queue.cc:1035] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f4e5215283d4a398bad003c1e63b434" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 39599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:03:54.106155 18961 tablet.cc:2369] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:03:54.116065 19010 mvcc.cc:205] Tried to move back new op lower bound from 7316281074052673536 to 7316281073851097088. Current Snapshot: MvccSnapshot[applied={T|T < 7316281074052673536}]
I20260808 15:03:54.129920 18220 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:54.130192 18220 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:03:54.133667 18220 catalog_manager.cc:5782] T 00000000000000000000000000000000 P df5ebe701fb54d69b1d9eac601e21117: Sending DeleteTablet for 3 replicas of tablet 766609a06c4242f7b1f3409b63cf0c82
I20260808 15:03:54.134721 18851 tablet_service.cc:1514] Processing DeleteTablet for tablet 766609a06c4242f7b1f3409b63cf0c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:54 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33282
I20260808 15:03:54.134743 18652 tablet_service.cc:1514] Processing DeleteTablet for tablet 766609a06c4242f7b1f3409b63cf0c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:54 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33898
I20260808 15:03:54.134840 18419 tablet_service.cc:1514] Processing DeleteTablet for tablet 766609a06c4242f7b1f3409b63cf0c82 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:03:54 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:51230
I20260808 15:03:54.140810 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 18318
I20260808 15:03:54.145856 19028 tablet_replica.cc:330] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: stopping tablet replica
I20260808 15:03:54.146301 19028 raft_consensus.cc:2229] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:54.146883 19028 raft_consensus.cc:2258] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:54.147621 19026 tablet_replica.cc:330] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: stopping tablet replica
I20260808 15:03:54.147876 19027 tablet_replica.cc:330] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: stopping tablet replica
I20260808 15:03:54.148044 19026 raft_consensus.cc:2229] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:03:54.148286 19027 raft_consensus.cc:2229] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:03:54.148550 19026 raft_consensus.cc:2258] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:54.148940 19027 raft_consensus.cc:2258] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:03:54.149857 19028 ts_tablet_manager.cc:1905] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:54.150357 19026 ts_tablet_manager.cc:1905] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:54.151052 19027 ts_tablet_manager.cc:1905] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:03:54.156673 19028 ts_tablet_manager.cc:1918] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:54.156867 19028 log.cc:1196] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/766609a06c4242f7b1f3409b63cf0c82
I20260808 15:03:54.156996 19026 ts_tablet_manager.cc:1918] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:54.157227 19026 log.cc:1196] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/766609a06c4242f7b1f3409b63cf0c82
I20260808 15:03:54.157357 19027 ts_tablet_manager.cc:1918] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:03:54.157442 19028 ts_tablet_manager.cc:1939] T 766609a06c4242f7b1f3409b63cf0c82 P 1af3b9b15f594749b8b10e3c789a7dec: Deleting consensus metadata
I20260808 15:03:54.157510 19027 log.cc:1196] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/766609a06c4242f7b1f3409b63cf0c82
I20260808 15:03:54.157927 19026 ts_tablet_manager.cc:1939] T 766609a06c4242f7b1f3409b63cf0c82 P 7f4e5215283d4a398bad003c1e63b434: Deleting consensus metadata
I20260808 15:03:54.159219 18205 catalog_manager.cc:4841] TS 1af3b9b15f594749b8b10e3c789a7dec (127.2.180.193:33439): tablet 766609a06c4242f7b1f3409b63cf0c82 (table test-table [id=55ba4a7c89a44803b2596a118e8a2d2b]) successfully deleted
Aug 08 15:03:54 dist-test-slave-jlzn krb5kdc[18161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:03:54.159638 18206 catalog_manager.cc:4841] TS 7f4e5215283d4a398bad003c1e63b434 (127.2.180.195:39599): tablet 766609a06c4242f7b1f3409b63cf0c82 (table test-table [id=55ba4a7c89a44803b2596a118e8a2d2b]) successfully deleted
I20260808 15:03:54.160192 19027 ts_tablet_manager.cc:1939] T 766609a06c4242f7b1f3409b63cf0c82 P df6d90904a22473c926159db2cee1b24: Deleting consensus metadata
I20260808 15:03:54.161638 18207 catalog_manager.cc:4841] TS df6d90904a22473c926159db2cee1b24 (127.2.180.194:43283): tablet 766609a06c4242f7b1f3409b63cf0c82 (table test-table [id=55ba4a7c89a44803b2596a118e8a2d2b]) successfully deleted
I20260808 15:03:54.166832 18986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:54.142314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51236 (local address 127.2.180.193:33439)
0808 15:03:54.142477 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:54.142483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:54.149642 (+ 7159us) server_negotiation.cc:407] Connection header received
0808 15:03:54.149906 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:54.149912 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:54.149989 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:54.150063 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:54.151751 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.152874 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.154591 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.155265 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.155362 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:54.160548 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:54.160563 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:54.160579 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:54.160618 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:54.164139 (+ 3521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:54.164754 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:54.164763 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:54.164767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:54.164930 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:54.165326 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:54.165332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:54.165335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:54.165789 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:54.166100 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:54.166255 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 15:03:54.166466 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:03:54.167451 18522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:54.254714 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 18318
I20260808 15:03:54.280525 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 18538
Aug 08 15:03:54 dist-test-slave-jlzn krb5kdc[18161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:03:54.295706 18980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:54.281383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33920 (local address 127.2.180.194:43283)
0808 15:03:54.281606 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:54.281611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:54.281632 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:03:54.281785 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:54.281790 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:54.281874 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:54.281973 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:54.283156 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.284005 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.285225 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.285678 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.285796 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:54.290052 (+ 4256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:54.290072 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:54.290088 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:54.290130 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:54.293100 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:54.293860 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:54.293867 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:54.293869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:54.294019 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:54.294370 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:54.294373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:54.294375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:54.294716 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:54.294973 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:54.295261 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 15:03:54.295412 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 15:03:54.296464 18746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:54.412578 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 18538
I20260808 15:03:54.436410 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 18761
Aug 08 15:03:54 dist-test-slave-jlzn krb5kdc[18161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:03:54.450698 18984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:03:54.437139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.2.180.195:39599)
0808 15:03:54.437319 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:03:54.437325 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:03:54.437345 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:03:54.437521 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:03:54.437525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:03:54.437602 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:03:54.437684 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:03:54.439058 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.439972 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.441229 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:03:54.441611 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:03:54.441710 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:03:54.445850 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:03:54.445864 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:03:54.445882 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:03:54.445916 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:03:54.448532 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:54.449103 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:54.449109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:54.449112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:54.449254 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:03:54.449527 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:03:54.449530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:03:54.449532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:03:54.449869 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:03:54.450085 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 15:03:54.450289 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 15:03:54.450448 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:03:54.451300 18951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:54.536811 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 18761
I20260808 15:03:54.559994 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 18181
I20260808 15:03:54.561053 18269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:03:54.667253 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 18181
2026-08-08T15:03:54Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4221 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:03:54 dist-test-slave-jlzn krb5kdc[19043](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:03:54 dist-test-slave-jlzn krb5kdc[19043](info): set up 2 sockets
Aug 08 15:03:54 dist-test-slave-jlzn krb5kdc[19043](info): commencing operation
krb5kdc: starting...
W20260808 15:03:55.874469 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:03:56 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201436, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:03:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:03:56Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.180.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.180.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/rangeradmin_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/rangeradmin_127.2.180.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.180.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.180.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/rangerlookup_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/rangerlookup_127.2.180.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.180.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.180.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/HTTP_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/HTTP_127.2.180.212@KRBTEST.COM.keytab.
I20260808 15:03:56.176060 2771 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6Pln5Y/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/postgres -l logfile start
2026-08-08 15:03:58.390 UTC [19098] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 15:03:58.390 UTC [19098] LOG: listening on IPv4 address "127.2.180.212", port 42965
2026-08-08 15:03:58.393 UTC [19098] LOG: listening on Unix socket "/tmp/.s.PGSQL.42965"
2026-08-08 15:03:58.416 UTC [19101] LOG: database system was shut down at 2026-08-08 15:03:57 UTC
2026-08-08 15:03:58.419 UTC [19098] LOG: database system is ready to accept connections
I20260808 15:03:59.469133 2771 mini_postgres.cc:96] Postgres bound to 42965
2026-08-08 15:03:59.491 UTC [19116] FATAL: database "slave" does not exist
127.2.180.212:42965 - accepting connections
I20260808 15:03:59.492632 2771 mini_ranger.cc:161] Starting Ranger...
I20260808 15:03:59.519695 2771 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 15:03:59.910859 2771 mini_ranger.cc:87] Created ranger Postgres database
I20260808 15:03:59.911015 2771 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-admin
2026-08-08 15:04:00,243 [I] DB FLAVOR :POSTGRES
2026-08-08 15:04:00,243 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 15:04:00,244 [I] Checking connection..
2026-08-08 15:04:00,244 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 15:04:00,481 [I] Checking connection passed.
2026-08-08 15:04:00,481 [I] --------- Verifying version history table ---------
2026-08-08 15:04:00,481 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:04:00,728 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 15:04:00,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:04:00,988 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 15:04:00,989 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 15:04:00,989 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 15:04:01.229 UTC [19207] WARNING: there is no transaction in progress
2026-08-08 15:04:01,236 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 15:04:01,237 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:04:01,485 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 15:04:01,486 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 15:04:01,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 15:04:01,737 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 15:04:01,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jlzn.c.gcp-upstream.internal','N') ;"
2026-08-08 15:04:02,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 15:04:02,441 [I] Table x_portal_user does not exist in database ranger
2026-08-08 15:04:02,441 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 15:04:02,695 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 15:04:02,695 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 15:04:02,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 15:04:03,175 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 15:04:03,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 15:04:04.036 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.050 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.064 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.077 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.091 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.106 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.165 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.171 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.184 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.193 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.202 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.214 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.222 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.227 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.232 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04.238 UTC [19365] WARNING: there is no transaction in progress
2026-08-08 15:04:04,750 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 15:04:04,750 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 15:04:04,985 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 15:04:04,986 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 15:04:05,224 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 15:04:05,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 15:04:05,474 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 15:04:05,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 15:04:05,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 15:04:05,923 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jlzn.c.gcp-upstream.internal';"
2026-08-08 15:04:06,147 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 15:04:06,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 15:04:06,396 [I] DB_PATCHES have already been applied
I20260808 15:04:06.402262 2771 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-admin:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6Pln5Y/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 15:04:06.402349 2771 mini_ranger.cc:193] Using host: 127.2.180.212
I20260808 15:04:06.421260 2771 mini_ranger.cc:227] Ranger admin URL: http://127.2.180.212:45697
Aug 08, 2026 3:04:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:04:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:04:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 3:04:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 3:04:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 3:04:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 3:04:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 15:04:07.597666 2771 mini_ranger.cc:160] Time spent starting Ranger: real 8.105s user 0.010s sys 0.040s
Aug 08, 2026 3:04:25 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 3:04:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 3:04:25 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 3:04:26 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 3:04:26 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 15:04:27.577293 2771 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.180.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.180.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/rangerkms_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/rangerkms_127.2.180.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.180.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.180.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.180.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/HTTP_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.180.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/HTTP_127.2.180.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 15:04:27 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201467, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 15:04:27.745147 2771 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 15:04:27.764650 2771 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 15:04:28.475543 2771 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 15:04:28.475715 2771 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms
I20260808 15:04:28.475837 2771 mini_ranger_kms.cc:227] Using postgres at 127.2.180.212:42965
2026-08-08 15:04:28,965 [I] DB FLAVOR :POSTGRES
2026-08-08 15:04:28,966 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 15:04:28,966 [I] Checking connection
2026-08-08 15:04:28,966 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 15:04:29,217 [I] connection success
2026-08-08 15:04:29,217 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 15:04:29,217 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 15:04:29,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 15:04:29,449 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 15:04:29,449 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 15:04:29,449 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 15:04:29,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:42965/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 15:04:29,716 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 15:04:30.136171 2771 mini_ranger_kms.cc:310] Created kms service
I20260808 15:04:30.311327 2771 mini_ranger_kms.cc:326] Created kudu user
I20260808 15:04:30.350405 2771 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 15:04:30 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201467, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:04:30.724715 2771 mini_ranger_kms.cc:384] Added ranger policy
I20260808 15:04:30.725138 2771 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms:/tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 15:04:30.725209 2771 mini_ranger_kms.cc:240] Using host: 127.2.180.212
I20260808 15:04:30.749758 2771 mini_ranger_kms.cc:276] Ranger KMS PID: 19633
I20260808 15:04:30.749934 2771 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.180.212:35045
Aug 08, 2026 3:04:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:04:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:04:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 3:04:31 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 3:04:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 3:04:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 15:04:32 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786201472, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.180.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:04:33.193559 2771 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.448s user 0.120s sys 0.400s
I20260808 15:04:33.193945 2771 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 15:04:33.194181 2771 mini_ranger_kms.cc:401] 127.2.180.212:35045/kms/v1/keys
Aug 08 15:04:33 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201472, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.180.212@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
Aug 08 15:04:33 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201472, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.180.212@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
Aug 08 15:04:36 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:04:36.227020 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:44451
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:44451
--ranger_config_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:35045/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:04:36.551304 19704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:04:36.551585 19704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:04:36.551707 19704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:04:36.558545 19704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:04:36.558635 19704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:04:36.558692 19704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:04:36.558751 19704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:04:36.558799 19704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:04:36.568329 19704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.180.212:35045/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:44451
--ranger_config_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:44451
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:04:36.569413 19704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:04:36.570540 19704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:04:36.581519 19713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:04:36.581585 19711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:04:36.581610 19715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:04:36.582697 19704 server_base.cc:1034] running on GCE node
I20260808 15:04:36.583384 19704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:04:36.584687 19704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:04:36.585877 19704 hybrid_clock.cc:648] HybridClock initialized: now 1786201476585838 us; error 30 us; skew 500 ppm
I20260808 15:04:36.586236 19704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:04:36 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201476, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:04:36.589123 19704 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:04:36.590404 19704 webserver.cc:460] Webserver started at http://127.2.180.254:44481/ using document root <none> and password file <none>
I20260808 15:04:36.590963 19704 fs_manager.cc:359] Metadata directory not provided
I20260808 15:04:36.591073 19704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:04:36.591349 19704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:04:36 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201476, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:04:36.650027 19704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "e47f04a3cf86470fa849d858f2113672"
format_stamp: "Formatted at 2026-08-08 15:04:36 on dist-test-slave-jlzn"
server_key: "e78ef382a572a13b87db57a18684bce4"
server_key_iv: "ead4856f343e314a2a877e79f95f42ff"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:36.651175 19704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "e47f04a3cf86470fa849d858f2113672"
format_stamp: "Formatted at 2026-08-08 15:04:36 on dist-test-slave-jlzn"
server_key: "e78ef382a572a13b87db57a18684bce4"
server_key_iv: "ead4856f343e314a2a877e79f95f42ff"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:36.658209 19704 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260808 15:04:36.677127 19726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:36.678881 19704 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:04:36.679068 19704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "e47f04a3cf86470fa849d858f2113672"
format_stamp: "Formatted at 2026-08-08 15:04:36 on dist-test-slave-jlzn"
server_key: "e78ef382a572a13b87db57a18684bce4"
server_key_iv: "ead4856f343e314a2a877e79f95f42ff"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:36.679322 19704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:04:36.708055 19704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:04:36.711784 19704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:04:36.712098 19704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:04:36.730502 19704 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:44451
I20260808 15:04:36.730576 19812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:44451 every 8 connection(s)
I20260808 15:04:36.731776 19704 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:04:36.736295 19813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:04:36.739192 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 19704
I20260808 15:04:36.739374 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:04:36.740058 2771 external_mini_cluster.cc:1428] Setting key cda4d9a88f588b11adf17d8bacae96ce
I20260808 15:04:36.750783 19813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672: Bootstrap starting.
I20260808 15:04:36.754490 19813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672: Neither blocks nor log segments found. Creating new log.
I20260808 15:04:36.755785 19813 log.cc:824] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672: Log is configured to *not* fsync() on all Append() calls
I20260808 15:04:36.760380 19813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672: No bootstrap required, opened a new log
Aug 08 15:04:36 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:04:36.771274 19813 raft_consensus.cc:348] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f04a3cf86470fa849d858f2113672" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44451 } }
I20260808 15:04:36.771783 19813 raft_consensus.cc:374] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:04:36.771919 19813 raft_consensus.cc:729] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e47f04a3cf86470fa849d858f2113672, State: Initialized, Role: FOLLOWER
I20260808 15:04:36.772585 19813 consensus_queue.cc:260] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f04a3cf86470fa849d858f2113672" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44451 } }
I20260808 15:04:36.772876 19813 raft_consensus.cc:388] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:04:36.773013 19813 raft_consensus.cc:482] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:04:36.773192 19813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:04:36.776003 19816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:36.742045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60596 (local address 127.2.180.254:44451)
0808 15:04:36.757382 (+ 15337us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:36.757398 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:36.757466 (+ 68us) server_negotiation.cc:407] Connection header received
0808 15:04:36.759359 (+ 1893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:36.759381 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:36.759738 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:36.760244 (+ 506us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:04:36.761835 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:36.763184 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:36.764507 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:36.765189 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:36.765348 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:36.768936 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:04:36.768944 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:04:36.768960 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:04:36.768994 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:04:36.771771 (+ 2777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:36.772596 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:36.772603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:36.772608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:36.772753 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:36.773254 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:36.773259 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:36.773262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:36.773813 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:04:36.774143 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:36.774534 (+ 391us) server_negotiation.cc:299] Negotiation successful
0808 15:04:36.774939 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15099,"thread_start_us":14852,"threads_started":1}
I20260808 15:04:36.776697 19813 raft_consensus.cc:504] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f04a3cf86470fa849d858f2113672" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44451 } }
I20260808 15:04:36.777287 19813 leader_election.cc:302] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e47f04a3cf86470fa849d858f2113672; no voters:
I20260808 15:04:36.777982 19813 leader_election.cc:288] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:04:36.778153 19824 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:04:36.780073 19824 raft_consensus.cc:686] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [term 1 LEADER]: Becoming Leader. State: Replica: e47f04a3cf86470fa849d858f2113672, State: Running, Role: LEADER
I20260808 15:04:36.780714 19824 consensus_queue.cc:237] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f04a3cf86470fa849d858f2113672" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44451 } }
I20260808 15:04:36.782755 19824 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e47f04a3cf86470fa849d858f2113672. Latest consensus state: current_term: 1 leader_uuid: "e47f04a3cf86470fa849d858f2113672" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f04a3cf86470fa849d858f2113672" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44451 } } }
I20260808 15:04:36.783094 19824 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [sys.catalog]: This master's current role is: LEADER
I20260808 15:04:36.783553 19826 sys_catalog.cc:455] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e47f04a3cf86470fa849d858f2113672" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e47f04a3cf86470fa849d858f2113672" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44451 } } }
I20260808 15:04:36.783902 19826 sys_catalog.cc:458] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [sys.catalog]: This master's current role is: LEADER
I20260808 15:04:36.786345 19813 sys_catalog.cc:564] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:04:36.789903 19829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:04:36.794850 19829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:04:36.818687 19829 catalog_manager.cc:1347] Generated new cluster ID: 0958773ecfd345a793603fd486efae98
I20260808 15:04:36.818826 19829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:04:36.830987 19813 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 15:04:36.858158 19829 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:04:36.859556 19829 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:04:36.870332 19829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672: Generated new TSK 0
I20260808 15:04:36.871368 19829 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:04:37.629348 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44451
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:04:37 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786201477, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 15:04:38.014814 19887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:04:38.015184 19887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:04:38.015344 19887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:04:38.026367 19887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:04:38.026496 19887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:04:38.026688 19887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:04:38.041579 19887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.180.212:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:04:38.043196 19887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:04:38.044967 19887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:04:38.062690 19902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:04:38.064802 19903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:04:38.072630 19905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:04:38.074404 19887 server_base.cc:1034] running on GCE node
I20260808 15:04:38.075181 19887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:04:38.076187 19887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:04:38.077414 19887 hybrid_clock.cc:648] HybridClock initialized: now 1786201478077361 us; error 45 us; skew 500 ppm
I20260808 15:04:38.077749 19887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:04:38 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201478, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:04:38.081354 19887 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:04:38.088971 19887 webserver.cc:460] Webserver started at http://127.2.180.193:44281/ using document root <none> and password file <none>
I20260808 15:04:38.089638 19887 fs_manager.cc:359] Metadata directory not provided
I20260808 15:04:38.089756 19887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:04:38.090062 19887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:04:38 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201478, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:04:38.116261 19887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d"
format_stamp: "Formatted at 2026-08-08 15:04:38 on dist-test-slave-jlzn"
server_key: "fdf9cb0b2f0d59f8ec1ee8f1915d4b7f"
server_key_iv: "6107d83dd8244b85116fc3187387a118"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:38.117518 19887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d"
format_stamp: "Formatted at 2026-08-08 15:04:38 on dist-test-slave-jlzn"
server_key: "fdf9cb0b2f0d59f8ec1ee8f1915d4b7f"
server_key_iv: "6107d83dd8244b85116fc3187387a118"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:38.123293 19887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 15:04:38.149477 19919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:38.151351 19887 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.005s sys 0.004s
I20260808 15:04:38.151566 19887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d"
format_stamp: "Formatted at 2026-08-08 15:04:38 on dist-test-slave-jlzn"
server_key: "fdf9cb0b2f0d59f8ec1ee8f1915d4b7f"
server_key_iv: "6107d83dd8244b85116fc3187387a118"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:38.151873 19887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:04:38.188935 19887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:04:38.193792 19887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:04:38.194144 19887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:04:38.196105 19887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:04:38.197949 19887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:04:38.198053 19887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:38.198195 19887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:04:38.198262 19887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 15:04:38 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201477, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:04:38.446785 19887 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:33995
I20260808 15:04:38.446785 20086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:33995 every 8 connection(s)
I20260808 15:04:38.448654 19887 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:04:38.456902 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 19887
I20260808 15:04:38.457130 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:04:38.458034 2771 external_mini_cluster.cc:1428] Setting key d7d3e121052773d2c634c2dbbb776155
Aug 08 15:04:38 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201478, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
I20260808 15:04:38.510663 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:38.452193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:50517 (local address 127.2.180.254:44451)
0808 15:04:38.490381 (+ 38188us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:38.490387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:38.490413 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:04:38.491617 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:38.491625 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:38.491741 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:38.491935 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:04:38.494190 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:38.495337 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:38.497727 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:38.498359 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:38.498497 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:38.502802 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:04:38.502808 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:04:38.502813 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:04:38.502852 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:04:38.506314 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:38.507309 (+ 995us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:38.507314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:38.507318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:38.507473 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:38.508026 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:38.508031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:38.508034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:38.508415 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:04:38.508560 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:38.510185 (+ 1625us) server_negotiation.cc:299] Negotiation successful
0808 15:04:38.510388 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38048,"thread_start_us":40390,"threads_started":1}
I20260808 15:04:38.512108 20090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:38.453110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44451 (local address 127.2.180.193:50517)
0808 15:04:38.487986 (+ 34876us) negotiation.cc:107] Waiting for socket to connect
0808 15:04:38.488066 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 15:04:38.491096 (+ 3030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:04:38.492434 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:04:38.492449 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:04:38.493137 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:04:38.493990 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:38.494016 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:38.495527 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:38.495532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:38.497564 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:38.497579 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:38.498647 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:38.498656 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:38.498946 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:38.499710 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:04:38.499738 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:04:38.502542 (+ 2804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:04:38.506512 (+ 3970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:04:38.506522 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:04:38.506540 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:04:38.507142 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:04:38.507613 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:04:38.507618 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:04:38.507621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:04:38.507887 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:04:38.508640 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:04:38.508648 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:04:38.509100 (+ 452us) client_negotiation.cc:769] Sending connection context
0808 15:04:38.509422 (+ 322us) client_negotiation.cc:240] Negotiation successful
0808 15:04:38.510958 (+ 1536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34435,"thread_start_us":35773,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
I20260808 15:04:38.514062 20087 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44451
I20260808 15:04:38.514385 20087 heartbeater.cc:464] Registering TS with master...
I20260808 15:04:38.515220 20087 heartbeater.cc:511] Master 127.2.180.254:44451 requested a full tablet report, sending...
I20260808 15:04:38.523573 19757 ts_manager.cc:194] Registered new tserver with Master: 16b0e5e5f7584ba4a6b2142bb3c7a79d (127.2.180.193:33995)
I20260808 15:04:38.525343 19757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:50517
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:04:38.544916 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44451
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:04:38 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201477, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
W20260808 15:04:38.955330 20095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:04:38.955610 20095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:04:38.955775 20095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:04:38.963645 20095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:04:38.963768 20095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:04:38.963938 20095 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:04:38.973498 20095 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.180.212:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:04:38.974812 20095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:04:38.976112 20095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:04:38.988456 20115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:04:38.988413 20122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:04:38.988487 20117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:04:38.989776 20095 server_base.cc:1034] running on GCE node
I20260808 15:04:38.990324 20095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:04:38.991178 20095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:04:38.992380 20095 hybrid_clock.cc:648] HybridClock initialized: now 1786201478992337 us; error 27 us; skew 500 ppm
I20260808 15:04:38.992669 20095 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:04:38 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201478, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:04:38.995638 20095 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:04:38.996999 20095 webserver.cc:460] Webserver started at http://127.2.180.194:42597/ using document root <none> and password file <none>
I20260808 15:04:38.997632 20095 fs_manager.cc:359] Metadata directory not provided
I20260808 15:04:38.997747 20095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:04:38.998049 20095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:04:38 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201478, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:04:39.015535 20095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "317bf53369e64176a078492befbf9762"
format_stamp: "Formatted at 2026-08-08 15:04:39 on dist-test-slave-jlzn"
server_key: "22675641971305467b17d6e970668556"
server_key_iv: "37dcd9f3c8395b4617032001bc352988"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:39.016400 20095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "317bf53369e64176a078492befbf9762"
format_stamp: "Formatted at 2026-08-08 15:04:39 on dist-test-slave-jlzn"
server_key: "22675641971305467b17d6e970668556"
server_key_iv: "37dcd9f3c8395b4617032001bc352988"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:39.021592 20095 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 15:04:39.036001 20133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:39.037240 20095 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:04:39.037389 20095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "317bf53369e64176a078492befbf9762"
format_stamp: "Formatted at 2026-08-08 15:04:39 on dist-test-slave-jlzn"
server_key: "22675641971305467b17d6e970668556"
server_key_iv: "37dcd9f3c8395b4617032001bc352988"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:39.037575 20095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:04:39.055975 20095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:04:39.059367 20095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:04:39.059612 20095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:04:39.060456 20095 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:04:39.061653 20095 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:04:39.061725 20095 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:39.061817 20095 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:04:39.061861 20095 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:39.092132 20095 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:34479
I20260808 15:04:39.092200 20301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:34479 every 8 connection(s)
I20260808 15:04:39.093290 20095 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:04:39.103119 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 20095
I20260808 15:04:39.103298 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:04:39.103718 2771 external_mini_cluster.cc:1428] Setting key 084d7c6bbd392f6c513dfcc35a4caf7c
Aug 08 15:04:39 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201478, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:04:39.112684 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.095547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:51081 (local address 127.2.180.254:44451)
0808 15:04:39.096041 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.096048 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.096871 (+ 823us) server_negotiation.cc:407] Connection header received
0808 15:04:39.098575 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.098579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.098665 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.098762 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:04:39.100116 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.100970 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.102405 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.102916 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.103058 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.106606 (+ 3548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:04:39.106612 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:04:39.106616 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:04:39.106653 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:04:39.109728 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:39.110406 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:39.110411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:39.110415 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:39.110593 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:39.110989 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:39.110993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:39.110995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:39.111326 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:04:39.111474 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.112089 (+ 615us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.112322 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":254,"threads_started":1}
I20260808 15:04:39.113302 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.096052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44451 (local address 127.2.180.194:51081)
0808 15:04:39.096716 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 15:04:39.096751 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:04:39.098324 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:04:39.098944 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:04:39.098951 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:04:39.099335 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:04:39.099954 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.099972 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.101087 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.101090 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.102268 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.102281 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.103093 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.103098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.103328 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.104008 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:04:39.104039 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:04:39.106375 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:04:39.109855 (+ 3480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:04:39.109861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:04:39.109871 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:04:39.110282 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:04:39.110684 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:04:39.110687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:04:39.110690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:04:39.110884 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:04:39.111424 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:04:39.111429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:04:39.111701 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 15:04:39.111941 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 15:04:39.112289 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":260,"threads_started":1}
I20260808 15:04:39.114567 20302 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44451
I20260808 15:04:39.114850 20302 heartbeater.cc:464] Registering TS with master...
I20260808 15:04:39.115500 20302 heartbeater.cc:511] Master 127.2.180.254:44451 requested a full tablet report, sending...
I20260808 15:04:39.117023 19757 ts_manager.cc:194] Registered new tserver with Master: 317bf53369e64176a078492befbf9762 (127.2.180.194:34479)
I20260808 15:04:39.117789 19757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:51081
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:04:39.155761 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:35045/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44451
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:04:39.470484 20310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:04:39.470736 20310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:04:39.470865 20310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:04:39.477815 20310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:04:39.477911 20310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:04:39.478055 20310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:04:39.487419 20310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.180.212:35045/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44451
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:04:39.488538 20310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:04:39.489746 20310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:04:39.502178 20316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:04:39.502151 20318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:04:39.502208 20321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:04:39.508354 20310 server_base.cc:1034] running on GCE node
I20260808 15:04:39.508903 20310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:04:39.509750 20310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:04:39.510937 20310 hybrid_clock.cc:648] HybridClock initialized: now 1786201479510896 us; error 29 us; skew 500 ppm
I20260808 15:04:39.511250 20310 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:04:39 dist-test-slave-jlzn krb5kdc[19043](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201479, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:04:39.514324 20310 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:04:39.515692 20310 webserver.cc:460] Webserver started at http://127.2.180.195:42893/ using document root <none> and password file <none>
I20260808 15:04:39.516381 20310 fs_manager.cc:359] Metadata directory not provided
I20260808 15:04:39.516481 20310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:04:39.516759 20310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:04:39 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201479, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:04:39.528977 20087 heartbeater.cc:503] Master 127.2.180.254:44451 was elected leader, sending a full tablet report...
I20260808 15:04:39.532603 20310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "e3f80aeffb044064af6b2aa6efd8c6c2"
format_stamp: "Formatted at 2026-08-08 15:04:39 on dist-test-slave-jlzn"
server_key: "c50f41ef306563b59fbc6e00d6b6ed44"
server_key_iv: "a0ac80d251ff6e497782806bf0d0404f"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:39.533711 20310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "e3f80aeffb044064af6b2aa6efd8c6c2"
format_stamp: "Formatted at 2026-08-08 15:04:39 on dist-test-slave-jlzn"
server_key: "c50f41ef306563b59fbc6e00d6b6ed44"
server_key_iv: "a0ac80d251ff6e497782806bf0d0404f"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:39.540467 20310 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260808 15:04:39.556265 20330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:39.557687 20310 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.003s
I20260808 15:04:39.557833 20310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "e3f80aeffb044064af6b2aa6efd8c6c2"
format_stamp: "Formatted at 2026-08-08 15:04:39 on dist-test-slave-jlzn"
server_key: "c50f41ef306563b59fbc6e00d6b6ed44"
server_key_iv: "a0ac80d251ff6e497782806bf0d0404f"
server_key_version: "kuduclusterkey@0"
I20260808 15:04:39.558023 20310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:04:39.570420 20310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:04:39.574097 20310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:04:39.574384 20310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:04:39.575242 20310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:04:39.576565 20310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:04:39.576642 20310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:39.576764 20310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:04:39.576817 20310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:04:39.608901 20310 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:33485
I20260808 15:04:39.608935 20496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:33485 every 8 connection(s)
I20260808 15:04:39.610081 20310 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:04:39.614130 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 20310
I20260808 15:04:39.614298 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:04:39.614746 2771 external_mini_cluster.cc:1428] Setting key ef256bc51a4f499fb596442afc9cc76e
Aug 08 15:04:39 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201479, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:04:39.628782 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.612513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:49987 (local address 127.2.180.254:44451)
0808 15:04:39.612714 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.612719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.613819 (+ 1100us) server_negotiation.cc:407] Connection header received
0808 15:04:39.616120 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.616126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.616239 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.616326 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:04:39.617718 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.618454 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.620052 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.620469 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.620580 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.623613 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:04:39.623617 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:04:39.623621 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:04:39.623653 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:04:39.625938 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:39.626601 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:39.626607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:39.626610 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:39.626738 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:39.627152 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:39.627157 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:39.627160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:39.627411 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:04:39.627566 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.628281 (+ 715us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.628513 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:04:39.629863 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.612954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44451 (local address 127.2.180.195:49987)
0808 15:04:39.613671 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0808 15:04:39.613707 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:04:39.615840 (+ 2133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:04:39.616569 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:04:39.616577 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:04:39.616995 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:04:39.617554 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.617581 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.618616 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.618620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.619912 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.619925 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.620696 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.620704 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.620912 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.621456 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:04:39.621478 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:04:39.623421 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:04:39.626066 (+ 2645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:04:39.626074 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:04:39.626088 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:04:39.626489 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:04:39.626842 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:04:39.626847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:04:39.626851 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:04:39.627043 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:04:39.627534 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:04:39.627541 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:04:39.627894 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 15:04:39.628180 (+ 286us) client_negotiation.cc:240] Negotiation successful
0808 15:04:39.628639 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":300,"threads_started":1}
I20260808 15:04:39.630978 20497 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44451
I20260808 15:04:39.631239 20497 heartbeater.cc:464] Registering TS with master...
I20260808 15:04:39.631984 20497 heartbeater.cc:511] Master 127.2.180.254:44451 requested a full tablet report, sending...
I20260808 15:04:39.633303 19757 ts_manager.cc:194] Registered new tserver with Master: e3f80aeffb044064af6b2aa6efd8c6c2 (127.2.180.195:33485)
I20260808 15:04:39.634047 19757 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:49987
I20260808 15:04:39.641601 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:04:39.657435 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.645968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.2.180.254:44451)
0808 15:04:39.646139 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.646144 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.646621 (+ 477us) server_negotiation.cc:407] Connection header received
0808 15:04:39.646747 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.646751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.646807 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.646881 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:04:39.648181 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.648867 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.650140 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.650534 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.650615 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.652774 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:04:39.652778 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:04:39.652782 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:04:39.652810 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:04:39.655180 (+ 2370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:39.655724 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:39.655727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:39.655729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:39.655885 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:39.656212 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:39.656215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:39.656217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:39.656519 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:04:39.656764 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.656951 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.657122 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:04:39.660957 19757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:04:39.668182 19757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:04:39.691112 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.682194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.2.180.194:34479)
0808 15:04:39.684061 (+ 1867us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.684069 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.684094 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:04:39.684183 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.684189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.684406 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.684591 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:04:39.685188 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.686196 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.688853 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.690147 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.690287 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.690477 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.690658 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.690821 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1713,"thread_start_us":1600,"threads_started":1}
I20260808 15:04:39.691385 20515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.681788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:34479 (local address 127.0.0.1:33318)
0808 15:04:39.683268 (+ 1480us) negotiation.cc:107] Waiting for socket to connect
0808 15:04:39.683288 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:04:39.683413 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:04:39.684595 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:04:39.684600 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:04:39.684624 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:04:39.684995 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.685004 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.686335 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.686340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.688707 (+ 2367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.688719 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.690326 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.690330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.690451 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.690482 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:04:39.690587 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 15:04:39.690773 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1337,"mutex_wait_us":122,"thread_start_us":1075,"threads_started":1}
I20260808 15:04:39.692562 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.679971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:33995 (local address 127.0.0.1:59786)
0808 15:04:39.683139 (+ 3168us) negotiation.cc:107] Waiting for socket to connect
0808 15:04:39.683175 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:04:39.683372 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:04:39.684089 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:04:39.684094 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:04:39.684117 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:04:39.684618 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.684630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.686539 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.686544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.689034 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.689049 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.690978 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.690985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.691171 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.691204 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:04:39.691282 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 15:04:39.691398 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3041,"mutex_wait_us":22,"thread_start_us":1825,"threads_started":1}
I20260808 15:04:39.692552 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.680451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59786 (local address 127.2.180.193:33995)
0808 15:04:39.682691 (+ 2240us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.682698 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.683269 (+ 571us) server_negotiation.cc:407] Connection header received
0808 15:04:39.683578 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.683585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.683789 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.683990 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:04:39.684862 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.686391 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.689217 (+ 2826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.690601 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.690716 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.691988 (+ 1272us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.692064 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.692227 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2091,"thread_start_us":1929,"threads_started":1}
I20260808 15:04:39.693131 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.679855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:33485 (local address 127.0.0.1:37060)
0808 15:04:39.680588 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0808 15:04:39.680640 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 15:04:39.680828 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:04:39.683017 (+ 2189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:04:39.683025 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:04:39.683065 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:04:39.683529 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.683543 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.686196 (+ 2653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.686202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.689047 (+ 2845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.689056 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.692389 (+ 3333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.692395 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.692555 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.692586 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:04:39.692675 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 15:04:39.692792 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":97,"thread_start_us":332,"threads_started":1}
I20260808 15:04:39.693395 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.680123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37060 (local address 127.2.180.195:33485)
0808 15:04:39.682391 (+ 2268us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.682399 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.682432 (+ 33us) server_negotiation.cc:407] Connection header received
0808 15:04:39.682523 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.682531 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.682773 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.683596 (+ 823us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:04:39.684465 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.686059 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.690577 (+ 4518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.692185 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.692299 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.692506 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.692960 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.693091 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2113,"thread_start_us":1945,"threads_started":1}
I20260808 15:04:39.695307 20201 tablet_service.cc:1467] Processing CreateTablet for tablet 545713419b654d7c881aa48dc7b4b0e1 (DEFAULT_TABLE table=test-table [id=9a5c3d818d2e4203b359217e8f5d3881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:04:39.696393 20413 tablet_service.cc:1467] Processing CreateTablet for tablet 545713419b654d7c881aa48dc7b4b0e1 (DEFAULT_TABLE table=test-table [id=9a5c3d818d2e4203b359217e8f5d3881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:04:39.696843 20201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545713419b654d7c881aa48dc7b4b0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:04:39.697479 20413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545713419b654d7c881aa48dc7b4b0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:04:39.696985 19996 tablet_service.cc:1467] Processing CreateTablet for tablet 545713419b654d7c881aa48dc7b4b0e1 (DEFAULT_TABLE table=test-table [id=9a5c3d818d2e4203b359217e8f5d3881]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:04:39.698402 19996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 545713419b654d7c881aa48dc7b4b0e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:04:39.708636 20521 tablet_bootstrap.cc:492] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Bootstrap starting.
I20260808 15:04:39.711165 20526 tablet_bootstrap.cc:492] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Bootstrap starting.
I20260808 15:04:39.712549 20528 tablet_bootstrap.cc:492] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Bootstrap starting.
I20260808 15:04:39.714006 20521 tablet_bootstrap.cc:654] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Neither blocks nor log segments found. Creating new log.
I20260808 15:04:39.714551 20526 tablet_bootstrap.cc:654] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Neither blocks nor log segments found. Creating new log.
I20260808 15:04:39.715277 20528 tablet_bootstrap.cc:654] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Neither blocks nor log segments found. Creating new log.
I20260808 15:04:39.715302 20521 log.cc:824] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Log is configured to *not* fsync() on all Append() calls
I20260808 15:04:39.715732 20526 log.cc:824] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Log is configured to *not* fsync() on all Append() calls
I20260808 15:04:39.716408 20528 log.cc:824] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Log is configured to *not* fsync() on all Append() calls
I20260808 15:04:39.718526 20521 tablet_bootstrap.cc:492] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: No bootstrap required, opened a new log
I20260808 15:04:39.718529 20526 tablet_bootstrap.cc:492] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: No bootstrap required, opened a new log
I20260808 15:04:39.718794 20526 ts_tablet_manager.cc:1397] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 15:04:39.718798 20521 ts_tablet_manager.cc:1397] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260808 15:04:39.718907 20528 tablet_bootstrap.cc:492] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: No bootstrap required, opened a new log
I20260808 15:04:39.719177 20528 ts_tablet_manager.cc:1397] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:04:39.726298 20528 raft_consensus.cc:348] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.726632 20528 raft_consensus.cc:374] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:04:39.726738 20528 raft_consensus.cc:729] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16b0e5e5f7584ba4a6b2142bb3c7a79d, State: Initialized, Role: FOLLOWER
I20260808 15:04:39.727277 20528 consensus_queue.cc:260] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.728916 20528 ts_tablet_manager.cc:1428] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260808 15:04:39.728960 20521 raft_consensus.cc:348] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.729375 20521 raft_consensus.cc:374] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:04:39.729476 20521 raft_consensus.cc:729] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f80aeffb044064af6b2aa6efd8c6c2, State: Initialized, Role: FOLLOWER
I20260808 15:04:39.729242 20526 raft_consensus.cc:348] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.729615 20526 raft_consensus.cc:374] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:04:39.729713 20526 raft_consensus.cc:729] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 317bf53369e64176a078492befbf9762, State: Initialized, Role: FOLLOWER
I20260808 15:04:39.730054 20521 consensus_queue.cc:260] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.730237 20526 consensus_queue.cc:260] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.735105 20526 ts_tablet_manager.cc:1428] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Time spent starting tablet: real 0.016s user 0.014s sys 0.000s
I20260808 15:04:39.735483 20302 heartbeater.cc:503] Master 127.2.180.254:44451 was elected leader, sending a full tablet report...
I20260808 15:04:39.735984 20497 heartbeater.cc:503] Master 127.2.180.254:44451 was elected leader, sending a full tablet report...
I20260808 15:04:39.736614 20521 ts_tablet_manager.cc:1428] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Time spent starting tablet: real 0.018s user 0.012s sys 0.007s
W20260808 15:04:39.744222 20088 tablet.cc:2369] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:04:39.845100 20303 tablet.cc:2369] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:04:39.861874 20498 tablet.cc:2369] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:04:39.910027 20535 raft_consensus.cc:482] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:04:39.910306 20535 raft_consensus.cc:504] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.912046 20535 leader_election.cc:288] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 16b0e5e5f7584ba4a6b2142bb3c7a79d (127.2.180.193:33995), 317bf53369e64176a078492befbf9762 (127.2.180.194:34479)
I20260808 15:04:39.918325 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.912352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:52521 (local address 127.2.180.193:33995)
0808 15:04:39.912595 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.912601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.912684 (+ 83us) server_negotiation.cc:407] Connection header received
0808 15:04:39.912958 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.912964 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.913042 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.913147 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:04:39.913590 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.914437 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.916298 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.917417 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.917569 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.917638 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.917948 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.918072 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 15:04:39.918329 20499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.912260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:33995 (local address 127.2.180.195:52521)
0808 15:04:39.912574 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 15:04:39.912593 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:04:39.912734 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:04:39.913184 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:04:39.913190 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:04:39.913223 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:04:39.913442 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.913453 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.914561 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.914567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.916146 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.916165 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.917579 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.917585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.917763 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.917798 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 15:04:39.917901 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 15:04:39.918028 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188}
I20260808 15:04:39.918956 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.912765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:53607 (local address 127.2.180.194:34479)
0808 15:04:39.912990 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.912996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.913352 (+ 356us) server_negotiation.cc:407] Connection header received
0808 15:04:39.913493 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.913497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.913574 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.913665 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:04:39.914160 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.915039 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.917006 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.918030 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.918167 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.918235 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.918650 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.918757 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 15:04:39.919584 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.912538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:34479 (local address 127.2.180.195:53607)
0808 15:04:39.913231 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 15:04:39.913248 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:04:39.913385 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:04:39.913690 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:04:39.913693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:04:39.913713 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:04:39.914005 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.914013 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.915197 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.915203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.916853 (+ 1650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:04:39.916869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.918262 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.918268 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:04:39.918438 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.918484 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 15:04:39.919152 (+ 668us) client_negotiation.cc:240] Negotiation successful
0808 15:04:39.919284 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":381,"threads_started":1}
I20260808 15:04:39.919471 20029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545713419b654d7c881aa48dc7b4b0e1" candidate_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" is_pre_election: true
I20260808 15:04:39.920050 20029 raft_consensus.cc:2454] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3f80aeffb044064af6b2aa6efd8c6c2 in term 0.
I20260808 15:04:39.920595 20231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545713419b654d7c881aa48dc7b4b0e1" candidate_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317bf53369e64176a078492befbf9762" is_pre_election: true
I20260808 15:04:39.920686 20339 leader_election.cc:302] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0e5e5f7584ba4a6b2142bb3c7a79d, e3f80aeffb044064af6b2aa6efd8c6c2; no voters:
I20260808 15:04:39.921008 20535 raft_consensus.cc:2781] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:04:39.921005 20231 raft_consensus.cc:2454] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3f80aeffb044064af6b2aa6efd8c6c2 in term 0.
I20260808 15:04:39.921126 20535 raft_consensus.cc:482] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:04:39.921218 20535 raft_consensus.cc:3037] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:04:39.923955 20535 raft_consensus.cc:504] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.924521 20535 leader_election.cc:288] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [CANDIDATE]: Term 1 election: Requested vote from peers 16b0e5e5f7584ba4a6b2142bb3c7a79d (127.2.180.193:33995), 317bf53369e64176a078492befbf9762 (127.2.180.194:34479)
I20260808 15:04:39.924917 20029 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545713419b654d7c881aa48dc7b4b0e1" candidate_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d"
I20260808 15:04:39.924965 20231 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "545713419b654d7c881aa48dc7b4b0e1" candidate_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "317bf53369e64176a078492befbf9762"
I20260808 15:04:39.925127 20231 raft_consensus.cc:3037] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:04:39.925124 20029 raft_consensus.cc:3037] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:04:39.928237 20231 raft_consensus.cc:2454] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3f80aeffb044064af6b2aa6efd8c6c2 in term 1.
I20260808 15:04:39.928246 20029 raft_consensus.cc:2454] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3f80aeffb044064af6b2aa6efd8c6c2 in term 1.
I20260808 15:04:39.928624 20339 leader_election.cc:302] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16b0e5e5f7584ba4a6b2142bb3c7a79d, e3f80aeffb044064af6b2aa6efd8c6c2; no voters:
I20260808 15:04:39.928902 20535 raft_consensus.cc:2781] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:04:39.929342 20535 raft_consensus.cc:686] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 1 LEADER]: Becoming Leader. State: Replica: e3f80aeffb044064af6b2aa6efd8c6c2, State: Running, Role: LEADER
I20260808 15:04:39.929813 20535 consensus_queue.cc:237] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } }
I20260808 15:04:39.933945 19756 catalog_manager.cc:5495] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3f80aeffb044064af6b2aa6efd8c6c2 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f80aeffb044064af6b2aa6efd8c6c2" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 33485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 } health_report { overall_health: UNKNOWN } } }
I20260808 15:04:39.987041 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:39.981604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37066 (local address 127.2.180.195:33485)
0808 15:04:39.981791 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:39.981797 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:39.981926 (+ 129us) server_negotiation.cc:407] Connection header received
0808 15:04:39.982130 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:39.982134 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:39.982221 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:39.982345 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:04:39.982725 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.983418 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.984984 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:39.985378 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:39.985485 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:39.985967 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:04:39.986473 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:04:39.986598 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:39.986700 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 15:04:39.986775 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:04:39.994410 20029 raft_consensus.cc:1261] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 1 FOLLOWER]: Refusing update from remote peer e3f80aeffb044064af6b2aa6efd8c6c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:04:39.994408 20231 raft_consensus.cc:1261] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 1 FOLLOWER]: Refusing update from remote peer e3f80aeffb044064af6b2aa6efd8c6c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:04:39.995318 20535 consensus_queue.cc:1035] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16b0e5e5f7584ba4a6b2142bb3c7a79d" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 33995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:04:39.995596 20540 consensus_queue.cc:1035] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "317bf53369e64176a078492befbf9762" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 34479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:04:40.006520 20543 mvcc.cc:205] Tried to move back new op lower bound from 7316281262048391168 to 7316281261795942400. Current Snapshot: MvccSnapshot[applied={T|T < 7316281262048391168}]
I20260808 15:04:40.009809 20546 mvcc.cc:205] Tried to move back new op lower bound from 7316281262048391168 to 7316281261795942400. Current Snapshot: MvccSnapshot[applied={T|T < 7316281262048391168}]
I20260808 15:04:40.020470 19756 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:04:40.020707 19756 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60610:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:04:40.023883 19756 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e47f04a3cf86470fa849d858f2113672: Sending DeleteTablet for 3 replicas of tablet 545713419b654d7c881aa48dc7b4b0e1
I20260808 15:04:40.025039 19996 tablet_service.cc:1514] Processing DeleteTablet for tablet 545713419b654d7c881aa48dc7b4b0e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:04:40 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:59786
I20260808 15:04:40.025027 20413 tablet_service.cc:1514] Processing DeleteTablet for tablet 545713419b654d7c881aa48dc7b4b0e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:04:40 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:37060
I20260808 15:04:40.025172 20201 tablet_service.cc:1514] Processing DeleteTablet for tablet 545713419b654d7c881aa48dc7b4b0e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:04:40 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33318
I20260808 15:04:40.033366 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 19887
I20260808 15:04:40.035599 20557 tablet_replica.cc:330] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: stopping tablet replica
I20260808 15:04:40.036158 20557 raft_consensus.cc:2229] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:04:40.036929 20557 raft_consensus.cc:2258] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:04:40.038964 20557 ts_tablet_manager.cc:1905] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:04:40.043219 20554 tablet_replica.cc:330] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: stopping tablet replica
I20260808 15:04:40.043655 20554 raft_consensus.cc:2229] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:04:40.044386 20554 raft_consensus.cc:2258] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:04:40.047571 20553 tablet_replica.cc:330] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: stopping tablet replica
I20260808 15:04:40.047988 20553 raft_consensus.cc:2229] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:04:40.048221 20557 ts_tablet_manager.cc:1918] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:04:40.048432 20557 log.cc:1196] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/545713419b654d7c881aa48dc7b4b0e1
I20260808 15:04:40.049027 20557 ts_tablet_manager.cc:1939] T 545713419b654d7c881aa48dc7b4b0e1 P 317bf53369e64176a078492befbf9762: Deleting consensus metadata
I20260808 15:04:40.049778 20554 ts_tablet_manager.cc:1905] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 15:04:40 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:04:40.050859 19729 catalog_manager.cc:4841] TS 317bf53369e64176a078492befbf9762 (127.2.180.194:34479): tablet 545713419b654d7c881aa48dc7b4b0e1 (table test-table [id=9a5c3d818d2e4203b359217e8f5d3881]) successfully deleted
I20260808 15:04:40.051644 20553 raft_consensus.cc:2258] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:04:40.053715 20553 ts_tablet_manager.cc:1905] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:04:40.057128 20554 ts_tablet_manager.cc:1918] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:04:40.057327 20554 log.cc:1196] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/545713419b654d7c881aa48dc7b4b0e1
I20260808 15:04:40.057860 20554 ts_tablet_manager.cc:1939] T 545713419b654d7c881aa48dc7b4b0e1 P e3f80aeffb044064af6b2aa6efd8c6c2: Deleting consensus metadata
I20260808 15:04:40.059463 19729 catalog_manager.cc:4841] TS e3f80aeffb044064af6b2aa6efd8c6c2 (127.2.180.195:33485): tablet 545713419b654d7c881aa48dc7b4b0e1 (table test-table [id=9a5c3d818d2e4203b359217e8f5d3881]) successfully deleted
I20260808 15:04:40.060169 20517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:40.034794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59798 (local address 127.2.180.193:33995)
0808 15:04:40.034964 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:40.034971 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:40.040158 (+ 5187us) server_negotiation.cc:407] Connection header received
0808 15:04:40.040351 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:40.040357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:40.040447 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:40.040536 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:04:40.042534 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:40.043783 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:40.045847 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:40.046572 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:40.046676 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:40.052231 (+ 5555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:04:40.052245 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:04:40.052261 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:04:40.052306 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:04:40.056218 (+ 3912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:40.057179 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:40.057187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:40.057190 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:40.057537 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:40.058078 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:40.058083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:40.058087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:40.058603 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:04:40.058923 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:40.059513 (+ 590us) server_negotiation.cc:299] Negotiation successful
0808 15:04:40.059874 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:04:40.060757 20553 ts_tablet_manager.cc:1918] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:04:40.060938 20553 log.cc:1196] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/545713419b654d7c881aa48dc7b4b0e1
I20260808 15:04:40.061457 20082 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:04:40.061550 20553 ts_tablet_manager.cc:1939] T 545713419b654d7c881aa48dc7b4b0e1 P 16b0e5e5f7584ba4a6b2142bb3c7a79d: Deleting consensus metadata
I20260808 15:04:40.150347 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 19887
I20260808 15:04:40.150635 19730 catalog_manager.cc:4841] TS 16b0e5e5f7584ba4a6b2142bb3c7a79d (127.2.180.193:33995): tablet 545713419b654d7c881aa48dc7b4b0e1 (table test-table [id=9a5c3d818d2e4203b359217e8f5d3881]) successfully deleted
I20260808 15:04:40.179648 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 20095
Aug 08 15:04:40 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:04:40.195776 20518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:40.180437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33320 (local address 127.2.180.194:34479)
0808 15:04:40.180645 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:40.180651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:40.180671 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:04:40.180810 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:40.180816 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:40.180881 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:40.180963 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:04:40.182221 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:40.182962 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:40.184394 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:40.184821 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:40.184950 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:40.190202 (+ 5252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:04:40.190216 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:04:40.190233 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:04:40.190267 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:04:40.193153 (+ 2886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:40.193969 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:40.193977 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:40.193979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:40.194126 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:40.194481 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:40.194484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:40.194486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:40.194819 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:04:40.195037 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:40.195344 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 15:04:40.195496 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 15:04:40.196552 20296 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:04:40.301805 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 20095
I20260808 15:04:40.324399 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 20310
Aug 08 15:04:40 dist-test-slave-jlzn krb5kdc[19043](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:04:40.341846 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:04:40.325286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.2.180.195:33485)
0808 15:04:40.325450 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 15:04:40.325457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:04:40.325475 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:04:40.325690 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:04:40.325694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:04:40.325762 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:04:40.325848 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:04:40.327581 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:40.328424 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:40.330306 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:04:40.330755 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:04:40.330856 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:04:40.336165 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:04:40.336176 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:04:40.336192 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:04:40.336238 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:04:40.339114 (+ 2876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:40.339770 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:40.339777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:40.339779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:40.339990 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:04:40.340367 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:04:40.340371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:04:40.340373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:04:40.340715 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:04:40.341026 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 15:04:40.341281 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 15:04:40.341500 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:04:40.342499 20491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:04:40.459893 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 20310
I20260808 15:04:40.487236 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 19704
I20260808 15:04:40.488417 19806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:04:40.597777 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 19704
I20260808 15:04:40.628012 2771 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 15:04:40.972451 2771 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 15:04:40.972760 2771 mini_ranger.cc:66] Stopping Ranger...
I20260808 15:04:41.316697 2771 mini_ranger.cc:68] Stopped Ranger
2026-08-08 15:04:41.317 UTC [19098] LOG: received smart shutdown request
2026-08-08 15:04:41.320 UTC [19098] LOG: background worker "logical replication launcher" (PID 19114) exited with exit code 1
2026-08-08 15:04:41.320 UTC [19103] LOG: shutting down
2026-08-08 15:04:41.447 UTC [19098] LOG: database system is shut down
2026-08-08T15:04:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46873 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:04:41 dist-test-slave-jlzn krb5kdc[20575](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:04:41 dist-test-slave-jlzn krb5kdc[20575](info): set up 2 sockets
Aug 08 15:04:41 dist-test-slave-jlzn krb5kdc[20575](info): commencing operation
krb5kdc: starting...
W20260808 15:04:42.729519 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:04:42 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:04:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:04:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.2.180.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.2.180.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/rangeradmin_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/rangeradmin_127.2.180.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.2.180.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.2.180.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/rangerlookup_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/rangerlookup_127.2.180.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.180.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.2.180.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/HTTP_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/HTTP_127.2.180.212@KRBTEST.COM.keytab.
I20260808 15:04:43.036324 2771 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task6Pln5Y/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/postgres -l logfile start
2026-08-08 15:04:45.189 UTC [20616] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 15:04:45.189 UTC [20616] LOG: listening on IPv4 address "127.2.180.212", port 45421
2026-08-08 15:04:45.192 UTC [20616] LOG: listening on Unix socket "/tmp/.s.PGSQL.45421"
2026-08-08 15:04:45.211 UTC [20619] LOG: database system was shut down at 2026-08-08 15:04:44 UTC
2026-08-08 15:04:45.215 UTC [20616] LOG: database system is ready to accept connections
I20260808 15:04:46.262393 2771 mini_postgres.cc:96] Postgres bound to 45421
2026-08-08 15:04:46.286 UTC [20632] FATAL: database "slave" does not exist
127.2.180.212:45421 - accepting connections
I20260808 15:04:46.287353 2771 mini_ranger.cc:161] Starting Ranger...
I20260808 15:04:46.313361 2771 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 15:04:46.693027 2771 mini_ranger.cc:87] Created ranger Postgres database
I20260808 15:04:46.693156 2771 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-admin
2026-08-08 15:04:46,961 [I] DB FLAVOR :POSTGRES
2026-08-08 15:04:46,962 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 15:04:46,962 [I] Checking connection..
2026-08-08 15:04:46,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 15:04:47,216 [I] Checking connection passed.
2026-08-08 15:04:47,217 [I] --------- Verifying version history table ---------
2026-08-08 15:04:47,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:04:47,468 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 15:04:47,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:04:47,712 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 15:04:47,713 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 15:04:47,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 15:04:47.962 UTC [20738] WARNING: there is no transaction in progress
2026-08-08 15:04:47,970 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 15:04:47,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 15:04:48,213 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 15:04:48,213 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 15:04:48,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 15:04:48,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 15:04:48,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-jlzn.c.gcp-upstream.internal','N') ;"
2026-08-08 15:04:48,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 15:04:49,214 [I] Table x_portal_user does not exist in database ranger
2026-08-08 15:04:49,215 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 15:04:49,451 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 15:04:49,451 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 15:04:49,701 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 15:04:49,931 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 15:04:49,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 15:04:50.797 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.811 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.826 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.840 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.857 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.876 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.936 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.942 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.954 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.963 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.972 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.983 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.991 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:50.997 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:51.002 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:51.008 UTC [20898] WARNING: there is no transaction in progress
2026-08-08 15:04:51,546 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 15:04:51,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 15:04:51,789 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 15:04:51,789 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 15:04:52,031 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 15:04:52,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 15:04:52,283 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 15:04:52,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 15:04:52,527 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 15:04:52,768 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-jlzn.c.gcp-upstream.internal';"
2026-08-08 15:04:53,003 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 15:04:53,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 15:04:53,254 [I] DB_PATCHES have already been applied
I20260808 15:04:53.261034 2771 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-admin:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task6Pln5Y/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 15:04:53.261160 2771 mini_ranger.cc:193] Using host: 127.2.180.212
I20260808 15:04:53.279953 2771 mini_ranger.cc:227] Ranger admin URL: http://127.2.180.212:58071
Aug 08, 2026 3:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 3:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 3:04:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 3:04:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 3:04:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 15:04:54.461810 2771 mini_ranger.cc:160] Time spent starting Ranger: real 8.174s user 0.002s sys 0.045s
Aug 08, 2026 3:05:14 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 3:05:14 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 3:05:14 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 3:05:14 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 3:05:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 15:05:15.462406 2771 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.2.180.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.2.180.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/rangerkms_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.2.180.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/rangerkms_127.2.180.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.2.180.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.2.180.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.2.180.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/HTTP_127.2.180.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.2.180.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/HTTP_127.2.180.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 15:05:15 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201515, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 15:05:15.627950 2771 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 15:05:15.648304 2771 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 15:05:16.338063 2771 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 15:05:16.338182 2771 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms
I20260808 15:05:16.338232 2771 mini_ranger_kms.cc:227] Using postgres at 127.2.180.212:45421
2026-08-08 15:05:16,677 [I] DB FLAVOR :POSTGRES
2026-08-08 15:05:16,678 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 15:05:16,678 [I] Checking connection
2026-08-08 15:05:16,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 15:05:16,905 [I] connection success
2026-08-08 15:05:16,905 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 15:05:16,906 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 15:05:16,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 15:05:17,132 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 15:05:17,132 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 15:05:17,133 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 15:05:17,133 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.2.180.212:45421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 15:05:17,402 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 15:05:17.824885 2771 mini_ranger_kms.cc:310] Created kms service
I20260808 15:05:17.997826 2771 mini_ranger_kms.cc:326] Created kudu user
I20260808 15:05:18.039217 2771 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 15:05:18 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201515, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:18.410099 2771 mini_ranger_kms.cc:384] Added ranger policy
I20260808 15:05:18.410421 2771 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms:/tmp/dist-test-task6Pln5Y/build/asan/bin/postgresql.jar:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task6Pln5Y/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 15:05:18.410487 2771 mini_ranger_kms.cc:240] Using host: 127.2.180.212
I20260808 15:05:18.432542 2771 mini_ranger_kms.cc:276] Ranger KMS PID: 21149
I20260808 15:05:18.432658 2771 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.2.180.212:45601
Aug 08, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 3:05:19 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 15:05:20 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786201520, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.180.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:20.908663 2771 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.281s user 0.057s sys 0.304s
I20260808 15:05:20.908912 2771 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 15:05:20.909121 2771 mini_ranger_kms.cc:401] 127.2.180.212:45601/kms/v1/keys
Aug 08 15:05:21 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201520, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.180.212@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
Aug 08 15:05:21 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201520, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.2.180.212@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
Aug 08 15:05:23 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:05:23.657219 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:43343
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:43343
--ranger_config_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:45601/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:23.978760 21222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:23.979012 21222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:23.979120 21222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:23.986361 21222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:23.986454 21222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:23.986521 21222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:23.986617 21222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:23.986685 21222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:23.996232 21222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
--ranger_kms_url=127.2.180.212:45601/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:43343
--ranger_config_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:43343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:23.997279 21222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:23.998492 21222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:24.009472 21228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:24.009557 21229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:24.009699 21231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:24.010461 21222 server_base.cc:1034] running on GCE node
I20260808 15:05:24.011173 21222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:24.012514 21222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:24.013710 21222 hybrid_clock.cc:648] HybridClock initialized: now 1786201524013667 us; error 35 us; skew 500 ppm
I20260808 15:05:24.014014 21222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:24 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:24.016935 21222 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:05:24.018275 21222 webserver.cc:460] Webserver started at http://127.2.180.254:39313/ using document root <none> and password file <none>
I20260808 15:05:24.018846 21222 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:24.018939 21222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:24.019191 21222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:05:24 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201524, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:24.073607 21222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "7569729b58cc403c93a96b860848e1d2"
format_stamp: "Formatted at 2026-08-08 15:05:24 on dist-test-slave-jlzn"
server_key: "a6572ee1c23a1ec4604bad5aa3c01722"
server_key_iv: "9f9e596ceb76546da0c5c5782dec4b23"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:24.074411 21222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "7569729b58cc403c93a96b860848e1d2"
format_stamp: "Formatted at 2026-08-08 15:05:24 on dist-test-slave-jlzn"
server_key: "a6572ee1c23a1ec4604bad5aa3c01722"
server_key_iv: "9f9e596ceb76546da0c5c5782dec4b23"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:24.079783 21222 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 15:05:24.094478 21242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:24.095793 21222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 15:05:24.095988 21222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "7569729b58cc403c93a96b860848e1d2"
format_stamp: "Formatted at 2026-08-08 15:05:24 on dist-test-slave-jlzn"
server_key: "a6572ee1c23a1ec4604bad5aa3c01722"
server_key_iv: "9f9e596ceb76546da0c5c5782dec4b23"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:24.096184 21222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:24.112790 21222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:24.116475 21222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:24.116788 21222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:24.135149 21222 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:43343
I20260808 15:05:24.135174 21314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:43343 every 8 connection(s)
I20260808 15:05:24.136425 21222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:05:24.137346 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 21222
I20260808 15:05:24.137557 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:05:24.138404 2771 external_mini_cluster.cc:1428] Setting key 8c7d04cbe81034ee4a61877089ea3d08
I20260808 15:05:24.141479 21319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:24.154569 21319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Bootstrap starting.
I20260808 15:05:24.157817 21319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:24.158936 21319 log.cc:824] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:05:24 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:24.161952 21319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: No bootstrap required, opened a new log
I20260808 15:05:24.170745 21323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:24.140632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44192 (local address 127.2.180.254:43343)
0808 15:05:24.141986 (+ 1354us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:24.142001 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:24.148429 (+ 6428us) server_negotiation.cc:407] Connection header received
0808 15:05:24.150808 (+ 2379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:24.150839 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:24.151331 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:24.151760 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:24.153307 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:24.155342 (+ 2035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:24.157222 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:24.158275 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:24.158446 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:24.162602 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:24.162615 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:24.162632 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:24.162686 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:24.166451 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:24.167141 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:24.167150 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:24.167162 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:24.167340 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:24.167670 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:24.167675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:24.167679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:24.168412 (+ 733us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:24.168736 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:24.169065 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 15:05:24.169436 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1124,"thread_start_us":913,"threads_started":1}
I20260808 15:05:24.171793 21319 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } }
I20260808 15:05:24.172175 21319 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:24.172271 21319 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7569729b58cc403c93a96b860848e1d2, State: Initialized, Role: FOLLOWER
I20260808 15:05:24.172851 21319 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } }
I20260808 15:05:24.173063 21319 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:05:24.173218 21319 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:05:24.173367 21319 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:24.175405 21319 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } }
I20260808 15:05:24.175912 21319 leader_election.cc:302] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7569729b58cc403c93a96b860848e1d2; no voters:
I20260808 15:05:24.176461 21319 leader_election.cc:288] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:05:24.176592 21325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:05:24.177284 21325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 1 LEADER]: Becoming Leader. State: Replica: 7569729b58cc403c93a96b860848e1d2, State: Running, Role: LEADER
I20260808 15:05:24.177805 21325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } }
I20260808 15:05:24.178292 21319 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:24.180698 21328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7569729b58cc403c93a96b860848e1d2. Latest consensus state: current_term: 1 leader_uuid: "7569729b58cc403c93a96b860848e1d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } } }
I20260808 15:05:24.180663 21326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7569729b58cc403c93a96b860848e1d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } } }
I20260808 15:05:24.181123 21328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:24.181123 21326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:24.183470 21333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:05:24.188506 21333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:05:24.192322 21319 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 15:05:24.207633 21333 catalog_manager.cc:1347] Generated new cluster ID: e55ec3784a294250833aedc5a5b502af
I20260808 15:05:24.207772 21333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:05:24.225989 21333 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:05:24.227342 21333 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:05:24.234814 21333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Generated new TSK 0
I20260808 15:05:24.235532 21333 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:05:24.937350 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:45601/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:43343
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:05:25 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786201525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 15:05:25.327637 21381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:25.327955 21381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:25.328109 21381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:25.335274 21381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:25.335388 21381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:25.335546 21381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:05:25.348918 21381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.2.180.212:45601/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:25.350497 21381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:25.352142 21381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:25.367933 21399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:25.373301 21400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:25.373839 21381 server_base.cc:1034] running on GCE node
W20260808 15:05:25.375325 21403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:25.376197 21381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:25.377882 21381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:25.379166 21381 hybrid_clock.cc:648] HybridClock initialized: now 1786201525379020 us; error 134 us; skew 500 ppm
I20260808 15:05:25.379572 21381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:25 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:25.382721 21381 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:05:25.384450 21381 webserver.cc:460] Webserver started at http://127.2.180.193:42033/ using document root <none> and password file <none>
I20260808 15:05:25.385133 21381 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:25.385254 21381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:25.385556 21381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:05:25 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:25.410148 21381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "2b5e0ced056c4795926b0cfd48078d44"
format_stamp: "Formatted at 2026-08-08 15:05:25 on dist-test-slave-jlzn"
server_key: "4a2ed3b6a8d7f18591d98971efe713d2"
server_key_iv: "1ef8dd3d497be3a04e44295f01ccc049"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:25.410998 21381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "2b5e0ced056c4795926b0cfd48078d44"
format_stamp: "Formatted at 2026-08-08 15:05:25 on dist-test-slave-jlzn"
server_key: "4a2ed3b6a8d7f18591d98971efe713d2"
server_key_iv: "1ef8dd3d497be3a04e44295f01ccc049"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:25.416450 21381 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 15:05:25.445891 21410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:25.452006 21381 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.006s sys 0.008s
I20260808 15:05:25.452301 21381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "2b5e0ced056c4795926b0cfd48078d44"
format_stamp: "Formatted at 2026-08-08 15:05:25 on dist-test-slave-jlzn"
server_key: "4a2ed3b6a8d7f18591d98971efe713d2"
server_key_iv: "1ef8dd3d497be3a04e44295f01ccc049"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:25.452665 21381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:25.484306 21381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:25.489856 21381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:25.490191 21381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:25.492218 21381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:25.495697 21381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:25.495833 21381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:25.495975 21381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:25.496057 21381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:25.643867 21381 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:46461
I20260808 15:05:25.643919 21579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:46461 every 8 connection(s)
I20260808 15:05:25.645548 21381 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:05:25.652158 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 21381
I20260808 15:05:25.652372 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:05:25.653184 2771 external_mini_cluster.cc:1428] Setting key 6004f99c82fddbafbbf3a35bc5cd39f8
Aug 08 15:05:25 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
Aug 08 15:05:25 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
I20260808 15:05:25.704638 21582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:25.657527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:57677 (local address 127.2.180.254:43343)
0808 15:05:25.682933 (+ 25406us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:25.682940 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:25.682963 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:05:25.683661 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:25.683667 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:25.683766 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:25.683884 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:25.685967 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:25.687134 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:25.690465 (+ 3331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:25.691127 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:25.691219 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:25.695477 (+ 4258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:25.695484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:25.695488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:25.695528 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:25.700292 (+ 4764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:25.701902 (+ 1610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:25.701909 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:25.701912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:25.702076 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:25.702649 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:25.702655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:25.702658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:25.702954 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:25.703056 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:25.703969 (+ 913us) server_negotiation.cc:299] Negotiation successful
0808 15:05:25.704215 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25276,"thread_start_us":25424,"threads_started":1}
I20260808 15:05:25.706732 21583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:25.658192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:43343 (local address 127.2.180.193:57677)
0808 15:05:25.676671 (+ 18479us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:25.676732 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:25.683255 (+ 6523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:25.684333 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:25.684344 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:25.684971 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:25.685765 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:25.685790 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:25.687324 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:25.687330 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:25.690299 (+ 2969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:25.690315 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:25.691583 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:25.691590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:25.691905 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:25.692571 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:25.692599 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:25.695220 (+ 2621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:25.700449 (+ 5229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:25.700457 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:25.700474 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:25.701032 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:25.702246 (+ 1214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:25.702250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:25.702254 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:25.702505 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:25.703143 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:25.703149 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:25.703471 (+ 322us) client_negotiation.cc:769] Sending connection context
0808 15:05:25.704844 (+ 1373us) client_negotiation.cc:240] Negotiation successful
0808 15:05:25.705414 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18158,"thread_start_us":17936,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
I20260808 15:05:25.711728 21580 heartbeater.cc:347] Connected to a master server at 127.2.180.254:43343
I20260808 15:05:25.712060 21580 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:25.712888 21580 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
I20260808 15:05:25.715528 21262 ts_manager.cc:194] Registered new tserver with Master: 2b5e0ced056c4795926b0cfd48078d44 (127.2.180.193:46461)
I20260808 15:05:25.717556 21262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:57677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:05:25.751605 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:45601/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:43343
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:05:25 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201525, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
W20260808 15:05:26.187167 21589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:26.187683 21589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:26.187907 21589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:26.199019 21589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:26.199149 21589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:26.199404 21589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:05:26.214988 21589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.2.180.212:45601/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:26.216910 21589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:26.218638 21589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:26.253378 21599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:26.255035 21601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:26.264092 21589 server_base.cc:1034] running on GCE node
W20260808 15:05:26.268118 21605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:26.269048 21589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:26.278460 21589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:26.286602 21589 hybrid_clock.cc:648] HybridClock initialized: now 1786201526286439 us; error 152 us; skew 500 ppm
I20260808 15:05:26.287068 21589 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:26 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:26.295145 21589 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:05:26.335989 21589 webserver.cc:460] Webserver started at http://127.2.180.194:45513/ using document root <none> and password file <none>
I20260808 15:05:26.337059 21589 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:26.337273 21589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:26.337741 21589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:05:26 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:26.358436 21589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "9dcac56672ba46b69d47450e5731f819"
format_stamp: "Formatted at 2026-08-08 15:05:26 on dist-test-slave-jlzn"
server_key: "c27e745b1b0f469142d0e3b76b5dd808"
server_key_iv: "1371ff9be5715c3c569a5812ab665550"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:26.359934 21589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "9dcac56672ba46b69d47450e5731f819"
format_stamp: "Formatted at 2026-08-08 15:05:26 on dist-test-slave-jlzn"
server_key: "c27e745b1b0f469142d0e3b76b5dd808"
server_key_iv: "1371ff9be5715c3c569a5812ab665550"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:26.370209 21589 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.002s sys 0.006s
I20260808 15:05:26.398903 21622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:26.401031 21589 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.007s
I20260808 15:05:26.401305 21589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "9dcac56672ba46b69d47450e5731f819"
format_stamp: "Formatted at 2026-08-08 15:05:26 on dist-test-slave-jlzn"
server_key: "c27e745b1b0f469142d0e3b76b5dd808"
server_key_iv: "1371ff9be5715c3c569a5812ab665550"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:26.401659 21589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:26.456010 21589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:26.461437 21589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:26.461841 21589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:26.469195 21589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:26.477294 21589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:26.477435 21589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 15:05:26.477620 21589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:26.477726 21589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:26.721202 21580 heartbeater.cc:503] Master 127.2.180.254:43343 was elected leader, sending a full tablet report...
I20260808 15:05:26.799484 21589 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:35247
I20260808 15:05:26.799507 21780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:35247 every 8 connection(s)
I20260808 15:05:26.801225 21589 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:05:26.801702 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 21589
I20260808 15:05:26.801868 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:05:26.802459 2771 external_mini_cluster.cc:1428] Setting key e8545e7131256cbb68fac99d4177f222
Aug 08 15:05:26 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201526, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
I20260808 15:05:26.846746 21790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:26.819890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:44515 (local address 127.2.180.254:43343)
0808 15:05:26.825807 (+ 5917us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:26.825814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:26.828008 (+ 2194us) server_negotiation.cc:407] Connection header received
0808 15:05:26.829440 (+ 1432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:26.829447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:26.829533 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:26.829614 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:26.831855 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:26.832957 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:26.835251 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:26.835859 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:26.835959 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:26.840319 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:26.840326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:26.840331 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:26.840371 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:26.843503 (+ 3132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:26.844242 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:26.844248 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:26.844252 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:26.844374 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:26.844800 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:26.844804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:26.844806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:26.845093 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:26.845228 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:26.846099 (+ 871us) server_negotiation.cc:299] Negotiation successful
0808 15:05:26.846326 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5784,"thread_start_us":5654,"threads_started":1}
I20260808 15:05:26.847857 21787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:26.819276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:43343 (local address 127.2.180.194:44515)
0808 15:05:26.826385 (+ 7109us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:26.826437 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:26.829113 (+ 2676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:26.830180 (+ 1067us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:26.830192 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:26.830800 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:26.831630 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:26.831657 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:26.833177 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:26.833182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:26.835091 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:26.835107 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:26.836324 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:26.836331 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:26.836618 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:26.837348 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:26.837377 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:26.840063 (+ 2686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:26.843639 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:26.843648 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:26.843665 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:26.844132 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:26.844481 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:26.844486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:26.844490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:26.844685 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:26.845240 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:26.845250 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:26.845666 (+ 416us) client_negotiation.cc:769] Sending connection context
0808 15:05:26.845969 (+ 303us) client_negotiation.cc:240] Negotiation successful
0808 15:05:26.846489 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6808,"thread_start_us":6555,"threads_started":1}
I20260808 15:05:26.849494 21783 heartbeater.cc:347] Connected to a master server at 127.2.180.254:43343
I20260808 15:05:26.849838 21783 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:26.850682 21783 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
I20260808 15:05:26.852317 21262 ts_manager.cc:194] Registered new tserver with Master: 9dcac56672ba46b69d47450e5731f819 (127.2.180.194:35247)
I20260808 15:05:26.853433 21262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:44515
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:05:26.879197 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:45601/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:43343
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:27.199708 21794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:27.199996 21794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:27.200139 21794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:27.207609 21794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:27.207703 21794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:27.207898 21794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:05:27.217273 21794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.2.180.212:45601/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:27.218428 21794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:27.219587 21794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:27.231748 21799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:27.231874 21800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:27.231879 21804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:27.232573 21794 server_base.cc:1034] running on GCE node
I20260808 15:05:27.233085 21794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:27.233906 21794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:27.235085 21794 hybrid_clock.cc:648] HybridClock initialized: now 1786201527235032 us; error 47 us; skew 500 ppm
I20260808 15:05:27.235375 21794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:27 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:27.241662 21794 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:05:27.242982 21794 webserver.cc:460] Webserver started at http://127.2.180.195:36069/ using document root <none> and password file <none>
I20260808 15:05:27.243613 21794 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:27.243729 21794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:27.244031 21794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 15:05:27 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:27.258864 21794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "cdc550d311fa484dac9baad8cbba7a44"
format_stamp: "Formatted at 2026-08-08 15:05:27 on dist-test-slave-jlzn"
server_key: "8aee9a4ee9269ba7dc9783887fd81fab"
server_key_iv: "8c78cb07a13a050b267034090f827d12"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:27.259732 21794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "cdc550d311fa484dac9baad8cbba7a44"
format_stamp: "Formatted at 2026-08-08 15:05:27 on dist-test-slave-jlzn"
server_key: "8aee9a4ee9269ba7dc9783887fd81fab"
server_key_iv: "8c78cb07a13a050b267034090f827d12"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:27.264858 21794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 15:05:27.279274 21819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:27.280817 21794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:05:27.280983 21794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "cdc550d311fa484dac9baad8cbba7a44"
format_stamp: "Formatted at 2026-08-08 15:05:27 on dist-test-slave-jlzn"
server_key: "8aee9a4ee9269ba7dc9783887fd81fab"
server_key_iv: "8c78cb07a13a050b267034090f827d12"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:27.281212 21794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:27.305464 21794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:27.309558 21794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:27.309825 21794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:27.310768 21794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:27.312283 21794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:27.312363 21794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:27.312484 21794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:27.312537 21794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:27.351789 21794 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:46413
I20260808 15:05:27.351828 22000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:46413 every 8 connection(s)
I20260808 15:05:27.353029 21794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:05:27.361212 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 21794
I20260808 15:05:27.361390 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:05:27.361961 2771 external_mini_cluster.cc:1428] Setting key a0c4b064c30cb18df6bda9a255f23581
Aug 08 15:05:27 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201527, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:27.372471 22004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:27.355562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:36163 (local address 127.2.180.254:43343)
0808 15:05:27.356004 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:27.356008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:27.357158 (+ 1150us) server_negotiation.cc:407] Connection header received
0808 15:05:27.358943 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:27.358949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:27.359026 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:27.359119 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:27.360528 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.361263 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.362967 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.363536 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.363645 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:27.367057 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:27.367062 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:27.367065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:27.367096 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:27.369653 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:27.370323 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:27.370327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:27.370329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:27.370462 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:27.370880 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:27.370884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:27.370886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:27.371129 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:27.371271 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:27.371906 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 15:05:27.372124 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":242,"threads_started":1}
I20260808 15:05:27.373260 22005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:27.356260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:43343 (local address 127.2.180.195:36163)
0808 15:05:27.357002 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:27.357045 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:27.358669 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:27.359347 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:27.359354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:27.359791 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:27.360366 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:27.360383 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.361444 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.361449 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:27.362828 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:27.362841 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.363757 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.363765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:27.364083 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:27.364705 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:27.364729 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:27.366849 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:27.369779 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:27.369788 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:27.369803 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:27.370195 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:27.370563 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:27.370569 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:27.370572 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:27.370766 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:27.371255 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:27.371263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:27.371543 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 15:05:27.371762 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 15:05:27.372252 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":298,"threads_started":1}
I20260808 15:05:27.374522 22002 heartbeater.cc:347] Connected to a master server at 127.2.180.254:43343
I20260808 15:05:27.374800 22002 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:27.375463 22002 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
I20260808 15:05:27.376932 21262 ts_manager.cc:194] Registered new tserver with Master: cdc550d311fa484dac9baad8cbba7a44 (127.2.180.195:46413)
I20260808 15:05:27.377720 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:05:27.377921 21262 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:36163
I20260808 15:05:27.379662 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 21381
Aug 08 15:05:27 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:05:27.397746 22007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:27.380618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.2.180.193:46461)
0808 15:05:27.381272 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:27.381279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:27.381573 (+ 294us) server_negotiation.cc:407] Connection header received
0808 15:05:27.381690 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:27.381698 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:27.381917 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:27.382075 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:27.383450 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.385056 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.386592 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.387428 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.387581 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:27.392461 (+ 4880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:27.392473 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:27.392488 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:27.392527 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:27.395554 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:27.396209 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:27.396215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:27.396218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:27.396359 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:27.396627 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:27.396630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:27.396632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:27.396911 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:27.397128 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:27.397305 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 15:05:27.397480 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":323,"threads_started":1}
I20260808 15:05:27.398834 21570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:27.480373 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 21381
I20260808 15:05:27.500895 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 21589
Aug 08 15:05:27 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:05:27.515976 22011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:27.501728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52894 (local address 127.2.180.194:35247)
0808 15:05:27.502284 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:27.502291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:27.502316 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:05:27.502377 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:27.502383 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:27.502601 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:27.502727 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:27.503991 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.504984 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.506246 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.506750 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.506839 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:27.511043 (+ 4204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:27.511054 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:27.511067 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:27.511097 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:27.513665 (+ 2568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:27.514255 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:27.514261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:27.514264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:27.514390 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:27.514668 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:27.514671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:27.514673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:27.514979 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:27.515215 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:27.515419 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 15:05:27.515605 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":291,"threads_started":1}
I20260808 15:05:27.517210 21775 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:27.596163 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 21589
I20260808 15:05:27.619412 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 21794
Aug 08 15:05:27 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:05:27.634883 22013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:27.620251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.2.180.195:46413)
0808 15:05:27.620809 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:27.620816 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:27.620843 (+ 27us) server_negotiation.cc:407] Connection header received
0808 15:05:27.620903 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:27.620909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:27.621118 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:27.621265 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:27.622611 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.623646 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.625144 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:27.625699 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:27.625814 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:27.629967 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:27.629977 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:27.629990 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:27.630026 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:27.632567 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:27.633133 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:27.633139 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:27.633142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:27.633269 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:27.633538 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:27.633541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:27.633543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:27.633857 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:27.634092 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:27.634347 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 15:05:27.634534 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":261,"threads_started":1}
I20260808 15:05:27.636143 21992 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:27.717420 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 21794
I20260808 15:05:27.737138 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 21222
I20260808 15:05:27.738314 21310 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:27.841943 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 21222
I20260808 15:05:27.871345 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:43343
--webserver_interface=127.2.180.254
--webserver_port=39313
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:43343
--ranger_config_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:45601/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.180.212:45601/kms with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:28.181895 22016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:28.182173 22016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:28.182296 22016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:28.189620 22016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:28.189780 22016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:28.189850 22016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:28.189921 22016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:28.189968 22016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:28.199457 22016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.180.212:45601/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:43343
--ranger_config_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:43343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=39313
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:28.200618 22016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:28.201845 22016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:28.212642 22022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:28.212716 22023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:28.212687 22029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:28.213557 22016 server_base.cc:1034] running on GCE node
I20260808 15:05:28.214294 22016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:28.215512 22016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:28.216707 22016 hybrid_clock.cc:648] HybridClock initialized: now 1786201528216664 us; error 30 us; skew 500 ppm
I20260808 15:05:28.217022 22016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:28 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201528, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:28.219851 22016 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:05:28.221159 22016 webserver.cc:460] Webserver started at http://127.2.180.254:39313/ using document root <none> and password file <none>
I20260808 15:05:28.221799 22016 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:28.221925 22016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 15:05:28 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201528, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:28.296952 22016 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 15:05:28.300465 22040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:28.302045 22016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:05:28.302255 22016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "7569729b58cc403c93a96b860848e1d2"
format_stamp: "Formatted at 2026-08-08 15:05:24 on dist-test-slave-jlzn"
server_key: "a6572ee1c23a1ec4604bad5aa3c01722"
server_key_iv: "9f9e596ceb76546da0c5c5782dec4b23"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:28.303539 22016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:28.331987 22016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:28.335412 22016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:28.335650 22016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:28.352567 22016 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:43343
I20260808 15:05:28.352594 22117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:43343 every 8 connection(s)
I20260808 15:05:28.353965 22016 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:05:28.361748 22118 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:05:28.363655 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 22016
I20260808 15:05:28.364687 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:45601/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:46461
--local_ip_for_outbound_sockets=127.2.180.193
--tserver_master_addrs=127.2.180.254:43343
--webserver_port=42033
--webserver_interface=127.2.180.193
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.180.212:45601/kms with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:05:28.365433 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Bootstrap starting.
I20260808 15:05:28.386169 22118 log.cc:824] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:28.397464 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:05:28.398037 22118 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Bootstrap complete.
I20260808 15:05:28.411173 22118 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } }
I20260808 15:05:28.412369 22118 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7569729b58cc403c93a96b860848e1d2, State: Initialized, Role: FOLLOWER
I20260808 15:05:28.413079 22118 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } }
I20260808 15:05:28.413343 22118 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:05:28.413471 22118 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:05:28.413645 22118 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:05:28.419008 22118 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } }
I20260808 15:05:28.419575 22118 leader_election.cc:302] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7569729b58cc403c93a96b860848e1d2; no voters:
I20260808 15:05:28.420265 22118 leader_election.cc:288] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 15:05:28.420382 22125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:05:28.421463 22125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [term 2 LEADER]: Becoming Leader. State: Replica: 7569729b58cc403c93a96b860848e1d2, State: Running, Role: LEADER
I20260808 15:05:28.421927 22125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } }
I20260808 15:05:28.422353 22118 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:28.424059 22127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7569729b58cc403c93a96b860848e1d2. Latest consensus state: current_term: 2 leader_uuid: "7569729b58cc403c93a96b860848e1d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } } }
I20260808 15:05:28.424418 22127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:28.424902 22126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7569729b58cc403c93a96b860848e1d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7569729b58cc403c93a96b860848e1d2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 43343 } } }
I20260808 15:05:28.425203 22126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:28.426265 22130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:05:28.431983 22130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:05:28.433377 22130 catalog_manager.cc:1259] Loaded cluster ID: e55ec3784a294250833aedc5a5b502af
I20260808 15:05:28.433473 22130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:05:28.437399 22130 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:05:28.438745 22130 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Loaded TSK: 0
I20260808 15:05:28.440233 22130 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:05:28.444844 22118 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 15:05:28.813352 22120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:28.813719 22120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:28.813877 22120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:28.823201 22120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:28.823334 22120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:28.823496 22120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:05:28.834266 22120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.180.212:45601/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:46461
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=42033
--tserver_master_addrs=127.2.180.254:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:28.835846 22120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:28.837388 22120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:28.862561 22173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:28.870133 22174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:28.871219 22176 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:28.876976 22120 server_base.cc:1034] running on GCE node
I20260808 15:05:28.877753 22120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:28.878787 22120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:28.880004 22120 hybrid_clock.cc:648] HybridClock initialized: now 1786201528879944 us; error 43 us; skew 500 ppm
I20260808 15:05:28.880313 22120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:28 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201528, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:28.883374 22120 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:05:28.901336 22120 webserver.cc:460] Webserver started at http://127.2.180.193:42033/ using document root <none> and password file <none>
I20260808 15:05:28.902338 22120 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:28.902521 22120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 15:05:28 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201528, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:28.937404 22120 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.006s sys 0.000s
I20260808 15:05:28.940407 22190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:28.942087 22120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:05:28.942272 22120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "2b5e0ced056c4795926b0cfd48078d44"
format_stamp: "Formatted at 2026-08-08 15:05:25 on dist-test-slave-jlzn"
server_key: "4a2ed3b6a8d7f18591d98971efe713d2"
server_key_iv: "1ef8dd3d497be3a04e44295f01ccc049"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:28.943619 22120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:28.964326 22120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:28.968570 22120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:28.968822 22120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:28.969770 22120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:28.971107 22120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:28.971201 22120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:28.971282 22120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:28.971349 22120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:29.019824 22120 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:46461
I20260808 15:05:29.019903 22349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:46461 every 8 connection(s)
I20260808 15:05:29.021242 22120 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:05:29.031854 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 22120
I20260808 15:05:29.032482 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:45601/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:35247
--local_ip_for_outbound_sockets=127.2.180.194
--tserver_master_addrs=127.2.180.254:43343
--webserver_port=45513
--webserver_interface=127.2.180.194
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.180.212:45601/kms with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:05:29 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201528, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:29.047794 22354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:29.024313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:50817 (local address 127.2.180.254:43343)
0808 15:05:29.025015 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:29.025029 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:29.025631 (+ 602us) server_negotiation.cc:407] Connection header received
0808 15:05:29.027724 (+ 2093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:29.027736 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:29.028164 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:29.028531 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:29.029927 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:29.031655 (+ 1728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:29.033993 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:29.035001 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:29.035156 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:29.039664 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:29.039674 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:29.039692 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:29.039739 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:29.043475 (+ 3736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:29.044291 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:29.044297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:29.044302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:29.044468 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:29.044875 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:29.044878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:29.044880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:29.045363 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:29.045539 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:29.046270 (+ 731us) server_negotiation.cc:299] Negotiation successful
0808 15:05:29.046642 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":273,"threads_started":1}
I20260808 15:05:29.047859 22355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:29.024583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:43343 (local address 127.2.180.193:50817)
0808 15:05:29.025462 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:29.025504 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:29.027189 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:29.028706 (+ 1517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:29.028716 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:29.029200 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:29.029730 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:29.029748 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:29.031839 (+ 2091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:29.031845 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:29.033816 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:29.033833 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:29.035597 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:29.035605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:29.035910 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:29.036626 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:29.036665 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:29.039381 (+ 2716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:29.043645 (+ 4264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:29.043665 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:29.043677 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:29.044142 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:29.044579 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:29.044583 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:29.044585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:29.044762 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:29.045494 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:29.045502 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:29.045790 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 15:05:29.046017 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 15:05:29.046516 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":491,"threads_started":1}
I20260808 15:05:29.051556 22351 heartbeater.cc:347] Connected to a master server at 127.2.180.254:43343
I20260808 15:05:29.051873 22351 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:29.052542 22351 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
W20260808 15:05:29.055306 22351 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:43343 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 15:05:29.055511 22351 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:29.055747 22351 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
I20260808 15:05:29.056756 22351 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:29.057000 22351 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
W20260808 15:05:29.057932 22351 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:05:29.453766 22357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:29.454020 22357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:29.454149 22357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:29.461155 22357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:29.461261 22357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:29.461400 22357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:05:29.470638 22357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.180.212:45601/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:35247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=45513
--tserver_master_addrs=127.2.180.254:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:29.471796 22357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:29.473153 22357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:29.489223 22373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:29.489369 22374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:29.489248 22376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:29.490455 22357 server_base.cc:1034] running on GCE node
I20260808 15:05:29.491140 22357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:29.496091 22357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:29.497319 22357 hybrid_clock.cc:648] HybridClock initialized: now 1786201529497277 us; error 68 us; skew 500 ppm
I20260808 15:05:29.497606 22357 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:29 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:29.500607 22357 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:05:29.502408 22357 webserver.cc:460] Webserver started at http://127.2.180.194:45513/ using document root <none> and password file <none>
I20260808 15:05:29.503022 22357 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:29.503152 22357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 15:05:29 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:29.528381 22357 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.002s sys 0.004s
I20260808 15:05:29.531152 22389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:29.532639 22357 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:05:29.532801 22357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "9dcac56672ba46b69d47450e5731f819"
format_stamp: "Formatted at 2026-08-08 15:05:26 on dist-test-slave-jlzn"
server_key: "c27e745b1b0f469142d0e3b76b5dd808"
server_key_iv: "1371ff9be5715c3c569a5812ab665550"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:29.534121 22357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:29.588850 22357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:29.593920 22357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:29.594240 22357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:29.596833 22357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:29.599009 22357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:29.599124 22357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:29.599225 22357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:29.599295 22357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:29.644768 22357 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:35247
I20260808 15:05:29.644928 22552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:35247 every 8 connection(s)
I20260808 15:05:29.646610 22357 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:05:29.654311 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 22357
I20260808 15:05:29.655104 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.2.180.212:45601/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:46413
--local_ip_for_outbound_sockets=127.2.180.195
--tserver_master_addrs=127.2.180.254:43343
--webserver_port=36069
--webserver_interface=127.2.180.195
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:39499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.2.180.212:45601/kms with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:05:29 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786201529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 15:05:29 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:29.692049 22557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:29.650644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:43343 (local address 127.2.180.194:33541)
0808 15:05:29.660602 (+ 9958us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:29.660651 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:29.662755 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:29.664082 (+ 1327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:29.664094 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:29.664622 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:29.665354 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:29.665380 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:29.666823 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:29.666828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:29.668581 (+ 1753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:29.668594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:29.669352 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:29.669360 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:29.669580 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:29.670360 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:29.670390 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:29.681708 (+ 11318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:29.687227 (+ 5519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:29.687237 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:29.687254 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:29.687895 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:29.688363 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:29.688369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:29.688373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:29.688633 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:29.689324 (+ 691us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:29.689332 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:29.689722 (+ 390us) client_negotiation.cc:769] Sending connection context
0808 15:05:29.689995 (+ 273us) client_negotiation.cc:240] Negotiation successful
0808 15:05:29.690556 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8913,"thread_start_us":9052,"threads_started":1}
I20260808 15:05:29.693709 22556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:29.649977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:33541 (local address 127.2.180.254:43343)
0808 15:05:29.663362 (+ 13385us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:29.663368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:29.663387 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:05:29.663448 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:29.663453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:29.663531 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:29.663644 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:29.665588 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:29.666623 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:29.668729 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:29.669138 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:29.669247 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:29.683420 (+ 14173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:29.683428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:29.683433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:29.683476 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:29.687044 (+ 3568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:29.688058 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:29.688064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:29.688079 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:29.688232 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:29.688788 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:29.688793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:29.688797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:29.689161 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:29.692971 (+ 3810us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:29.693127 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 15:05:29.693358 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13260,"thread_start_us":13858,"threads_started":1}
I20260808 15:05:29.694969 22554 heartbeater.cc:347] Connected to a master server at 127.2.180.254:43343
I20260808 15:05:29.695307 22554 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:29.696187 22554 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
I20260808 15:05:29.698084 22066 ts_manager.cc:194] Registered new tserver with Master: 9dcac56672ba46b69d47450e5731f819 (127.2.180.194:35247)
I20260808 15:05:29.700212 22066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:33541
W20260808 15:05:30.041554 22558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:30.041944 22558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:30.042107 22558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:30.053210 22558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:30.053334 22558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:30.053517 22558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:05:30.058090 22351 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:30.058495 22351 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
I20260808 15:05:30.059866 22066 ts_manager.cc:194] Registered new tserver with Master: 2b5e0ced056c4795926b0cfd48078d44 (127.2.180.193:46461)
I20260808 15:05:30.061069 22066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:50817
I20260808 15:05:30.068174 22558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:39499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.2.180.212:45601/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:46413
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=36069
--tserver_master_addrs=127.2.180.254:43343
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:30.069571 22558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:30.071094 22558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:30.094219 22568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:30.100412 22573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:30.102247 22558 server_base.cc:1034] running on GCE node
W20260808 15:05:30.102516 22576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:30.103266 22558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
Aug 08 15:05:30 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:30.110543 22558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:30.112165 22558 hybrid_clock.cc:648] HybridClock initialized: now 1786201530112092 us; error 53 us; skew 500 ppm
I20260808 15:05:30.112534 22558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:30 dist-test-slave-jlzn krb5kdc[20575](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201530, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:30.123690 22558 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:05:30.128458 22558 webserver.cc:460] Webserver started at http://127.2.180.195:36069/ using document root <none> and password file <none>
I20260808 15:05:30.129494 22558 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:30.129699 22558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 15:05:30 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201530, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:30.163026 22558 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 15:05:30.165766 22585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:30.167083 22558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 15:05:30.167241 22558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "cdc550d311fa484dac9baad8cbba7a44"
format_stamp: "Formatted at 2026-08-08 15:05:27 on dist-test-slave-jlzn"
server_key: "8aee9a4ee9269ba7dc9783887fd81fab"
server_key_iv: "8c78cb07a13a050b267034090f827d12"
server_key_version: "kuduclusterkey@0"
I20260808 15:05:30.168249 22558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:30.205799 22558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:30.210583 22558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:30.210861 22558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:30.212073 22558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:30.213784 22558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:30.213896 22558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:30.213999 22558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:30.214074 22558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:30.252712 22558 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:46413
I20260808 15:05:30.252791 22749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:46413 every 8 connection(s)
I20260808 15:05:30.254076 22558 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:05:30.261891 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 22558
Aug 08 15:05:30 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201530, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:30.304891 22752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.262962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.2.180.254:43343)
0808 15:05:30.283206 (+ 20244us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.283211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.283232 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:05:30.283320 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.283325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.283422 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.283548 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:30.287905 (+ 4357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.289033 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.291041 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.291723 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.292516 (+ 793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.296122 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:30.296129 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:30.296133 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:30.296171 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:30.301696 (+ 5525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.302564 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.302570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.302573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.302727 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.303197 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.303202 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.303205 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.303564 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:30.303845 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.304263 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.304475 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20124,"thread_start_us":19806,"threads_started":1}
I20260808 15:05:30.311758 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.256908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:44347 (local address 127.2.180.254:43343)
0808 15:05:30.282759 (+ 25851us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.282766 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.289961 (+ 7195us) server_negotiation.cc:407] Connection header received
0808 15:05:30.292760 (+ 2799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.292766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.292834 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.292903 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:30.295027 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.296182 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.298495 (+ 2313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.299142 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.299227 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.304731 (+ 5504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:30.304738 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:30.304743 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:30.304779 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:30.308240 (+ 3461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.309171 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.309176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.309179 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.309326 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.309871 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.309876 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.309879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.310226 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:30.310375 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.311165 (+ 790us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.311369 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25721,"thread_start_us":32231,"threads_started":1}
I20260808 15:05:30.312959 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.257573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:43343 (local address 127.2.180.195:44347)
0808 15:05:30.289806 (+ 32233us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.289857 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:30.292451 (+ 2594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:30.293445 (+ 994us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:30.293457 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:30.294081 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:30.294830 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.294853 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.296394 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.296399 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.298338 (+ 1939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.298354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.299575 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.299581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.299885 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.301952 (+ 2067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:30.301987 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:30.304445 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:30.308443 (+ 3998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:30.308451 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:30.308466 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:30.309044 (+ 578us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:30.309471 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:30.309476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:30.309480 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:30.309747 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:30.310349 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:30.310357 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:30.310751 (+ 394us) client_negotiation.cc:769] Sending connection context
0808 15:05:30.311038 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 15:05:30.311568 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31920,"thread_start_us":31699,"threads_started":1}
I20260808 15:05:30.314378 22750 heartbeater.cc:347] Connected to a master server at 127.2.180.254:43343
I20260808 15:05:30.314704 22750 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:30.315517 22750 heartbeater.cc:511] Master 127.2.180.254:43343 requested a full tablet report, sending...
I20260808 15:05:30.317137 22066 ts_manager.cc:194] Registered new tserver with Master: cdc550d311fa484dac9baad8cbba7a44 (127.2.180.195:46413)
I20260808 15:05:30.318216 22066 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:44347
I20260808 15:05:30.321264 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:05:30.346668 22759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.324786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44206 (local address 127.2.180.254:43343)
0808 15:05:30.324940 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.324946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.331447 (+ 6501us) server_negotiation.cc:407] Connection header received
0808 15:05:30.331620 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.331626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.331708 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.331843 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:30.333743 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.334735 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.336557 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.337213 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.337332 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.340415 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:30.340422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:30.340426 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:30.340465 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:30.343788 (+ 3323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.344611 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.344617 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.344620 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.344767 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.345183 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.345188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.345191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.345538 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:30.345768 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.346111 (+ 343us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.346329 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:05:30.352427 22066 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:05:30.366556 22066 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:05:30.417724 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.400527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48398)
0808 15:05:30.408400 (+ 7873us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.408593 (+ 193us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:30.408788 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:30.409704 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:30.409708 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:30.409735 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:30.410040 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.410051 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.412328 (+ 2277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.412333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.414479 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.414494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.416732 (+ 2238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.416739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.416936 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.416991 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 15:05:30.417120 (+ 129us) client_negotiation.cc:240] Negotiation successful
0808 15:05:30.417302 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7755,"mutex_wait_us":3,"thread_start_us":3909,"threads_started":1}
I20260808 15:05:30.418612 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.400248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.2.180.193:46461)
0808 15:05:30.404276 (+ 4028us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.404285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.408689 (+ 4404us) server_negotiation.cc:407] Connection header received
0808 15:05:30.408968 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.408976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.409187 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.409331 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:30.410232 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.412151 (+ 1919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.414675 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.416133 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.416250 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.416497 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.418106 (+ 1609us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.418279 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3860,"thread_start_us":3693,"threads_started":1}
Aug 08 15:05:30 dist-test-slave-jlzn krb5kdc[20575](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201529, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for HTTP/127.2.180.212@KRBTEST.COM
I20260808 15:05:30.423084 22259 tablet_service.cc:1467] Processing CreateTablet for tablet 1f5534527941476b9a58cf82b13c6772 (DEFAULT_TABLE table=test-table [id=1b44cf546fb64663b7e81e61faca3f74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:30.425050 22259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f5534527941476b9a58cf82b13c6772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:30.429981 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.398221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:46413 (local address 127.0.0.1:47102)
0808 15:05:30.404727 (+ 6506us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.404744 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:30.404863 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:30.421308 (+ 16445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:30.421313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:30.421340 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:30.421654 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.421666 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.424336 (+ 2670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.424341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.426338 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.426354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.429263 (+ 2909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.429270 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.429447 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.429480 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:05:30.429544 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 15:05:30.429659 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6371,"thread_start_us":4401,"threads_started":1}
I20260808 15:05:30.431419 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.398752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.2.180.195:46413)
0808 15:05:30.418948 (+ 20196us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.418956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.418992 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:05:30.419110 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.419118 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.421087 (+ 1969us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.421886 (+ 799us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:30.422285 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.424190 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.427497 (+ 3307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.429020 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.430334 (+ 1314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.430549 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.430835 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.430994 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20018,"thread_start_us":22100,"threads_started":1}
I20260808 15:05:30.434458 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.396539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52908 (local address 127.2.180.194:35247)
0808 15:05:30.397165 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.397175 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.403176 (+ 6001us) server_negotiation.cc:407] Connection header received
0808 15:05:30.404794 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.404803 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.405061 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.405245 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:30.409498 (+ 4253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.411096 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.430581 (+ 19485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.431987 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.432089 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.432261 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.433960 (+ 1699us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.434133 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":334,"threads_started":1}
I20260808 15:05:30.435161 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.395976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:35247 (local address 127.0.0.1:52908)
0808 15:05:30.403019 (+ 7043us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.403058 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:30.404104 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:30.407568 (+ 3464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:30.407575 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:30.407603 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:30.408127 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.408143 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.411278 (+ 3135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.411282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.430401 (+ 19119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.430416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.432443 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.432449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.432619 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.432647 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 15:05:30.434657 (+ 2010us) client_negotiation.cc:240] Negotiation successful
0808 15:05:30.434798 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6799,"mutex_wait_us":16501,"thread_start_us":12456,"threads_started":1}
I20260808 15:05:30.434921 22655 tablet_service.cc:1467] Processing CreateTablet for tablet 1f5534527941476b9a58cf82b13c6772 (DEFAULT_TABLE table=test-table [id=1b44cf546fb64663b7e81e61faca3f74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:30.436334 22655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f5534527941476b9a58cf82b13c6772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:30.442955 22464 tablet_service.cc:1467] Processing CreateTablet for tablet 1f5534527941476b9a58cf82b13c6772 (DEFAULT_TABLE table=test-table [id=1b44cf546fb64663b7e81e61faca3f74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:30.443984 22464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f5534527941476b9a58cf82b13c6772. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:30.445508 22783 tablet_bootstrap.cc:492] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Bootstrap starting.
I20260808 15:05:30.449170 22783 tablet_bootstrap.cc:654] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:30.450479 22783 log.cc:824] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:30.476356 22785 tablet_bootstrap.cc:492] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Bootstrap starting.
I20260808 15:05:30.479302 22783 tablet_bootstrap.cc:492] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: No bootstrap required, opened a new log
I20260808 15:05:30.479650 22783 ts_tablet_manager.cc:1397] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Time spent bootstrapping tablet: real 0.035s user 0.023s sys 0.009s
I20260808 15:05:30.489279 22785 tablet_bootstrap.cc:654] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:30.490087 22787 tablet_bootstrap.cc:492] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: Bootstrap starting.
I20260808 15:05:30.494762 22787 tablet_bootstrap.cc:654] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:30.495616 22783 raft_consensus.cc:348] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.496148 22783 raft_consensus.cc:374] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:30.496187 22785 log.cc:824] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:30.496282 22783 raft_consensus.cc:729] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b5e0ced056c4795926b0cfd48078d44, State: Initialized, Role: FOLLOWER
I20260808 15:05:30.497071 22783 consensus_queue.cc:260] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.498009 22787 log.cc:824] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:30.512892 22785 tablet_bootstrap.cc:492] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: No bootstrap required, opened a new log
I20260808 15:05:30.513263 22785 ts_tablet_manager.cc:1397] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Time spent bootstrapping tablet: real 0.037s user 0.007s sys 0.007s
I20260808 15:05:30.517829 22783 ts_tablet_manager.cc:1428] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Time spent starting tablet: real 0.038s user 0.017s sys 0.013s
I20260808 15:05:30.520388 22787 tablet_bootstrap.cc:492] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: No bootstrap required, opened a new log
I20260808 15:05:30.520709 22787 ts_tablet_manager.cc:1397] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: Time spent bootstrapping tablet: real 0.031s user 0.014s sys 0.000s
I20260808 15:05:30.522120 22351 heartbeater.cc:503] Master 127.2.180.254:43343 was elected leader, sending a full tablet report...
W20260808 15:05:30.523854 22352 tablet.cc:2369] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:05:30.533627 22785 raft_consensus.cc:348] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.535933 22785 raft_consensus.cc:374] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:30.537564 22787 raft_consensus.cc:348] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.538091 22785 raft_consensus.cc:729] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dcac56672ba46b69d47450e5731f819, State: Initialized, Role: FOLLOWER
I20260808 15:05:30.538918 22785 consensus_queue.cc:260] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.539662 22787 raft_consensus.cc:374] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:30.539844 22787 raft_consensus.cc:729] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdc550d311fa484dac9baad8cbba7a44, State: Initialized, Role: FOLLOWER
I20260808 15:05:30.540539 22787 consensus_queue.cc:260] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.543228 22793 raft_consensus.cc:482] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:05:30.543311 22750 heartbeater.cc:503] Master 127.2.180.254:43343 was elected leader, sending a full tablet report...
I20260808 15:05:30.544094 22793 raft_consensus.cc:504] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.549500 22787 ts_tablet_manager.cc:1428] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: Time spent starting tablet: real 0.029s user 0.011s sys 0.005s
I20260808 15:05:30.551605 22554 heartbeater.cc:503] Master 127.2.180.254:43343 was elected leader, sending a full tablet report...
I20260808 15:05:30.552346 22785 ts_tablet_manager.cc:1428] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Time spent starting tablet: real 0.039s user 0.011s sys 0.005s
I20260808 15:05:30.560647 22793 leader_election.cc:288] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9dcac56672ba46b69d47450e5731f819 (127.2.180.194:35247), 2b5e0ced056c4795926b0cfd48078d44 (127.2.180.193:46461)
I20260808 15:05:30.561753 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.550406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:35247 (local address 127.2.180.195:56023)
0808 15:05:30.550578 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.550612 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:30.550731 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:30.551856 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:30.551861 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:30.551888 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:30.552077 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.552090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.553569 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.553575 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.555635 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.555652 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.561038 (+ 5386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.561044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.561232 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.561267 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 15:05:30.561368 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 15:05:30.561495 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 15:05:30.562426 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.551112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:56023 (local address 127.2.180.194:35247)
0808 15:05:30.551309 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.551315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.551335 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:05:30.551402 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.551407 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.551551 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.551666 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:30.552252 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.553398 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.559462 (+ 6064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.560722 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.560861 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.560931 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.562039 (+ 1108us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.562171 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41,"spinlock_wait_cycles":36096}
I20260808 15:05:30.564704 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f5534527941476b9a58cf82b13c6772" candidate_uuid: "cdc550d311fa484dac9baad8cbba7a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dcac56672ba46b69d47450e5731f819" is_pre_election: true
I20260808 15:05:30.565196 22491 raft_consensus.cc:2454] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdc550d311fa484dac9baad8cbba7a44 in term 0.
I20260808 15:05:30.566042 22586 leader_election.cc:302] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dcac56672ba46b69d47450e5731f819, cdc550d311fa484dac9baad8cbba7a44; no voters:
I20260808 15:05:30.569830 22793 raft_consensus.cc:2781] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:05:30.570039 22793 raft_consensus.cc:482] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:05:30.570192 22793 raft_consensus.cc:3037] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:30.572456 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.556101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:38879 (local address 127.2.180.193:46461)
0808 15:05:30.556295 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.556301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.562991 (+ 6690us) server_negotiation.cc:407] Connection header received
0808 15:05:30.563223 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.563230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.563305 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.563361 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:30.566574 (+ 3213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.567647 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.569803 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.571056 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.571276 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.571358 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.571992 (+ 634us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.572128 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:05:30.573127 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.554938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.2.180.195:38879)
0808 15:05:30.562910 (+ 7972us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.562928 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:30.563081 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:30.563559 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:30.563564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:30.563589 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:30.563785 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.563795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.567850 (+ 4055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.567855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.569620 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:30.569635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.571524 (+ 1889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.571532 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:30.571764 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.571823 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 15:05:30.572673 (+ 850us) client_negotiation.cc:240] Negotiation successful
0808 15:05:30.572807 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7878,"thread_start_us":12052,"threads_started":1}
I20260808 15:05:30.573984 22793 raft_consensus.cc:504] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.574106 22287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f5534527941476b9a58cf82b13c6772" candidate_uuid: "cdc550d311fa484dac9baad8cbba7a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b5e0ced056c4795926b0cfd48078d44" is_pre_election: true
I20260808 15:05:30.574679 22287 raft_consensus.cc:2454] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cdc550d311fa484dac9baad8cbba7a44 in term 0.
I20260808 15:05:30.575090 22491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f5534527941476b9a58cf82b13c6772" candidate_uuid: "cdc550d311fa484dac9baad8cbba7a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dcac56672ba46b69d47450e5731f819"
I20260808 15:05:30.575280 22491 raft_consensus.cc:3037] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:30.578428 22491 raft_consensus.cc:2454] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdc550d311fa484dac9baad8cbba7a44 in term 1.
I20260808 15:05:30.578929 22586 leader_election.cc:302] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dcac56672ba46b69d47450e5731f819, cdc550d311fa484dac9baad8cbba7a44; no voters:
I20260808 15:05:30.579227 22792 raft_consensus.cc:2781] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:05:30.585453 22793 leader_election.cc:288] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [CANDIDATE]: Term 1 election: Requested vote from peers 9dcac56672ba46b69d47450e5731f819 (127.2.180.194:35247), 2b5e0ced056c4795926b0cfd48078d44 (127.2.180.193:46461)
I20260808 15:05:30.586071 22287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f5534527941476b9a58cf82b13c6772" candidate_uuid: "cdc550d311fa484dac9baad8cbba7a44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b5e0ced056c4795926b0cfd48078d44"
I20260808 15:05:30.586301 22287 raft_consensus.cc:3037] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:30.589454 22287 raft_consensus.cc:2454] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cdc550d311fa484dac9baad8cbba7a44 in term 1.
I20260808 15:05:30.589797 22792 raft_consensus.cc:686] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 1 LEADER]: Becoming Leader. State: Replica: cdc550d311fa484dac9baad8cbba7a44, State: Running, Role: LEADER
I20260808 15:05:30.590699 22792 consensus_queue.cc:237] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } }
I20260808 15:05:30.601806 22066 catalog_manager.cc:5495] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 reported cstate change: term changed from 0 to 1, leader changed from <none> to cdc550d311fa484dac9baad8cbba7a44 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "cdc550d311fa484dac9baad8cbba7a44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cdc550d311fa484dac9baad8cbba7a44" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46413 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 } health_report { overall_health: UNKNOWN } } }
W20260808 15:05:30.650710 22555 tablet.cc:2369] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:05:30.667586 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.659889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.2.180.195:46413)
0808 15:05:30.660065 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.660071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.660089 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:05:30.660173 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.660176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.660254 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.660384 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:05:30.660835 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.661953 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.664327 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.664965 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.665101 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.665833 (+ 732us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:05:30.666454 (+ 621us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:05:30.667171 (+ 717us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.667227 (+ 56us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.667313 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 15:05:30.677261 22287 raft_consensus.cc:1261] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 1 FOLLOWER]: Refusing update from remote peer cdc550d311fa484dac9baad8cbba7a44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:05:30.677268 22491 raft_consensus.cc:1261] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 1 FOLLOWER]: Refusing update from remote peer cdc550d311fa484dac9baad8cbba7a44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:05:30.678228 22792 consensus_queue.cc:1035] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b5e0ced056c4795926b0cfd48078d44" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 46461 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:05:30.678524 22793 consensus_queue.cc:1035] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dcac56672ba46b69d47450e5731f819" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:05:30.694823 22810 mvcc.cc:205] Tried to move back new op lower bound from 7316281469643067392 to 7316281469305286656. Current Snapshot: MvccSnapshot[applied={T|T < 7316281469643067392}]
I20260808 15:05:30.702920 22061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:05:30.703181 22061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:05:30.706503 22061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7569729b58cc403c93a96b860848e1d2: Sending DeleteTablet for 3 replicas of tablet 1f5534527941476b9a58cf82b13c6772
I20260808 15:05:30.707497 22464 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f5534527941476b9a58cf82b13c6772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:30 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:52908
I20260808 15:05:30.707543 22259 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f5534527941476b9a58cf82b13c6772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:30 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:48398
I20260808 15:05:30.707649 22655 tablet_service.cc:1514] Processing DeleteTablet for tablet 1f5534527941476b9a58cf82b13c6772 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:30 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:47102
I20260808 15:05:30.715619 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 22120
I20260808 15:05:30.720371 22823 tablet_replica.cc:330] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: stopping tablet replica
I20260808 15:05:30.720916 22823 raft_consensus.cc:2229] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:05:30.721751 22823 raft_consensus.cc:2258] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:30.722574 22826 tablet_replica.cc:330] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: stopping tablet replica
I20260808 15:05:30.722767 22825 tablet_replica.cc:330] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: stopping tablet replica
I20260808 15:05:30.723204 22825 raft_consensus.cc:2229] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:05:30.723730 22825 raft_consensus.cc:2258] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:30.724354 22823 ts_tablet_manager.cc:1905] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:05:30.726106 22825 ts_tablet_manager.cc:1905] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:05:30.726256 22826 raft_consensus.cc:2229] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:05:30.727278 22826 raft_consensus.cc:2258] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:30.729720 22826 ts_tablet_manager.cc:1905] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:05:30.731122 22823 ts_tablet_manager.cc:1918] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:05:30.731324 22823 log.cc:1196] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/1f5534527941476b9a58cf82b13c6772
I20260808 15:05:30.732014 22823 ts_tablet_manager.cc:1939] T 1f5534527941476b9a58cf82b13c6772 P 9dcac56672ba46b69d47450e5731f819: Deleting consensus metadata
I20260808 15:05:30.733762 22041 catalog_manager.cc:4841] TS 9dcac56672ba46b69d47450e5731f819 (127.2.180.194:35247): tablet 1f5534527941476b9a58cf82b13c6772 (table test-table [id=1b44cf546fb64663b7e81e61faca3f74]) successfully deleted
I20260808 15:05:30.736078 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.716474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48410 (local address 127.2.180.193:46461)
0808 15:05:30.717095 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.717101 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.717119 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:05:30.717173 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.717179 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.717266 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.717354 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:30.719501 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.720780 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.723234 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.723961 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.724090 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.727798 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:30.727830 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:30.727846 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:30.727892 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:30.731695 (+ 3803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.732575 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.732584 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.732587 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.732753 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.733208 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.733214 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.733217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.733686 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:30.734011 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.735398 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.735617 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486}
I20260808 15:05:30.736109 22825 ts_tablet_manager.cc:1918] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:05:30.736479 22825 log.cc:1196] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/1f5534527941476b9a58cf82b13c6772
I20260808 15:05:30.736791 22340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:30.740794 22826 ts_tablet_manager.cc:1918] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:05:30.741004 22826 log.cc:1196] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/1f5534527941476b9a58cf82b13c6772
I20260808 15:05:30.741607 22826 ts_tablet_manager.cc:1939] T 1f5534527941476b9a58cf82b13c6772 P cdc550d311fa484dac9baad8cbba7a44: Deleting consensus metadata
I20260808 15:05:30.743250 22043 catalog_manager.cc:4841] TS cdc550d311fa484dac9baad8cbba7a44 (127.2.180.195:46413): tablet 1f5534527941476b9a58cf82b13c6772 (table test-table [id=1b44cf546fb64663b7e81e61faca3f74]) successfully deleted
I20260808 15:05:30.825917 22825 ts_tablet_manager.cc:1939] T 1f5534527941476b9a58cf82b13c6772 P 2b5e0ced056c4795926b0cfd48078d44: Deleting consensus metadata
I20260808 15:05:30.826521 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 22120
I20260808 15:05:30.848701 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 22357
W20260808 15:05:30.848688 22042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.180.193:46461 (error 108)
I20260808 15:05:30.850607 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.850149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48426)
0808 15:05:30.850330 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.850459 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 15:05:30.850831 22042 catalog_manager.cc:4584] TS 2b5e0ced056c4795926b0cfd48078d44 (127.2.180.193:46461): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1f5534527941476b9a58cf82b13c6772: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
I20260808 15:05:30.868229 22775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.849536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52920 (local address 127.2.180.194:35247)
0808 15:05:30.849723 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.849729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.849851 (+ 122us) server_negotiation.cc:407] Connection header received
0808 15:05:30.850061 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.850067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.850137 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.850241 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:30.852256 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.852996 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.854230 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.854617 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.854729 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.861136 (+ 6407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:30.861150 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:30.861165 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:30.861200 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:30.864328 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.865204 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.865216 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.865220 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.865383 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.865820 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.865825 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.865829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.866358 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:30.866677 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:30.867214 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 15:05:30.867586 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:05:30.869288 22546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:30.904445 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.903915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48434)
0808 15:05:30.904142 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.904248 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 15:05:30.905603 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.905204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48448)
0808 15:05:30.905359 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.905463 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 15:05:30.962898 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 22357
I20260808 15:05:30.966750 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.966154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48450)
0808 15:05:30.966437 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.966542 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260808 15:05:30.968120 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.967627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48454)
0808 15:05:30.967871 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:30.967957 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260808 15:05:30.985797 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 22558
I20260808 15:05:31.001253 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:30.986696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47128 (local address 127.2.180.195:46413)
0808 15:05:30.986847 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:30.986852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:30.986874 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:05:30.986977 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:30.986981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:30.987053 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:30.987141 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:30.988915 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.989728 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.991567 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:30.992084 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:30.992185 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:30.995252 (+ 3067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:30.995265 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:30.995281 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:30.995321 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:30.998302 (+ 2981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.999132 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.999138 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.999140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:30.999288 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:30.999709 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:30.999712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:30.999714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:31.000127 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:31.000354 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:31.000743 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 15:05:31.000918 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:05:31.002010 22743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:31.048761 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:31.048205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48458)
0808 15:05:31.048437 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:31.048550 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 15:05:31.049857 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:31.049406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48474)
0808 15:05:31.049625 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:31.049716 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260808 15:05:31.123736 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 22558
I20260808 15:05:31.151939 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 22016
I20260808 15:05:31.153119 22113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:31.262260 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:31.261504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:46461 (local address 127.0.0.1:48476)
0808 15:05:31.261680 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:31.261808 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:46461: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 15:05:31.262635 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 22016
I20260808 15:05:31.291745 2771 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 15:05:31.635524 2771 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 15:05:31.635722 2771 mini_ranger.cc:66] Stopping Ranger...
I20260808 15:05:31.979142 2771 mini_ranger.cc:68] Stopped Ranger
2026-08-08 15:05:31.979 UTC [20616] LOG: received smart shutdown request
2026-08-08 15:05:31.981 UTC [20616] LOG: background worker "logical replication launcher" (PID 20628) exited with exit code 1
2026-08-08 15:05:31.982 UTC [20620] LOG: shutting down
2026-08-08 15:05:32.117 UTC [20616] LOG: database system is shut down
2026-08-08T15:05:32Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50674 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:05:32 dist-test-slave-jlzn krb5kdc[22846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:05:32 dist-test-slave-jlzn krb5kdc[22846](info): set up 2 sockets
Aug 08 15:05:32 dist-test-slave-jlzn krb5kdc[22846](info): commencing operation
krb5kdc: starting...
W20260808 15:05:33.396770 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:05:33 dist-test-slave-jlzn krb5kdc[22846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:05:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:05:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:05:33.615394 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45379
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45379,127.2.180.253:33253,127.2.180.252:41309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:33.932456 22867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:33.932726 22867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:33.932821 22867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:33.939709 22867 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:33.939827 22867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:33.939919 22867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:33.939991 22867 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:33.940049 22867 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:33.949383 22867 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45379,127.2.180.253:33253,127.2.180.252:41309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:33.950546 22867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:33.951767 22867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:33.962740 22879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:33.962838 22873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:33.962778 22875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:33.963868 22867 server_base.cc:1034] running on GCE node
I20260808 15:05:33.964586 22867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:33.965804 22867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:33.967026 22867 hybrid_clock.cc:648] HybridClock initialized: now 1786201533966956 us; error 42 us; skew 500 ppm
I20260808 15:05:33.967337 22867 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:33 dist-test-slave-jlzn krb5kdc[22846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201533, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:33.970265 22867 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:05:33.971547 22867 webserver.cc:460] Webserver started at http://127.2.180.254:41255/ using document root <none> and password file <none>
I20260808 15:05:33.972240 22867 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:33.972357 22867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:33.972627 22867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:05:33.975358 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b"
format_stamp: "Formatted at 2026-08-08 15:05:33 on dist-test-slave-jlzn"
server_key: "0e39e32a1331a54fb2a46cd10f20b48b"
server_key_iv: "5b620661c8d7cade79d8e5bc79af0750"
server_key_version: "encryptionkey@0"
I20260808 15:05:33.976166 22867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b"
format_stamp: "Formatted at 2026-08-08 15:05:33 on dist-test-slave-jlzn"
server_key: "0e39e32a1331a54fb2a46cd10f20b48b"
server_key_iv: "5b620661c8d7cade79d8e5bc79af0750"
server_key_version: "encryptionkey@0"
I20260808 15:05:33.980859 22867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:05:33.984004 22890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:33.985184 22867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:05:33.985428 22867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b"
format_stamp: "Formatted at 2026-08-08 15:05:33 on dist-test-slave-jlzn"
server_key: "0e39e32a1331a54fb2a46cd10f20b48b"
server_key_iv: "5b620661c8d7cade79d8e5bc79af0750"
server_key_version: "encryptionkey@0"
I20260808 15:05:33.985648 22867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:33.999348 22867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:34.002861 22867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:34.003098 22867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:34.019953 22867 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:45379
I20260808 15:05:34.019992 22960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:45379 every 8 connection(s)
I20260808 15:05:34.021113 22867 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:05:34.023011 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 22867
I20260808 15:05:34.023244 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:05:34.024117 2771 external_mini_cluster.cc:1428] Setting key 2413c900391b8f65988e46fb250a9ea1
I20260808 15:05:34.025103 22962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:34.033494 22962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201533, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.180.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.253@KRBTEST.COM" created.
I20260808 15:05:34.054405 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.036843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45379 (local address 127.0.0.1:40176)
0808 15:05:34.037572 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.037615 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:34.039511 (+ 1896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:34.040337 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:34.040344 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:34.040980 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:34.041299 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.041315 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.042810 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.042814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.043894 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.043907 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.044784 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.044791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.044966 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.045576 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:34.045598 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:34.047823 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:34.050773 (+ 2950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.050783 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.050790 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.051254 (+ 464us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.051688 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.051691 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.051694 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.051889 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.052762 (+ 873us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:34.052768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:34.052938 (+ 170us) client_negotiation.cc:769] Sending connection context
0808 15:05:34.053127 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 15:05:34.053454 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":288,"threads_started":1}
I20260808 15:05:34.054445 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.038048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.2.180.254:45379)
0808 15:05:34.038719 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:34.038726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:34.038774 (+ 48us) server_negotiation.cc:407] Connection header received
0808 15:05:34.039866 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:34.039872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:34.040118 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:34.040290 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:34.041479 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.042668 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.044025 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.044556 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.044719 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.048035 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:34.048042 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:34.048052 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:34.048081 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:34.050620 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.051387 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.051396 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.051400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.051572 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.051995 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.051999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.052002 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.052628 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:34.052810 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:34.053307 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 15:05:34.053602 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":479,"thread_start_us":304,"threads_started":1}
I20260808 15:05:34.057298 22962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:05:34.058336 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.057996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55604)
0808 15:05:34.058152 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.058245 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 15:05:34.058624 22898 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
I20260808 15:05:34.060143 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.059790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55606)
0808 15:05:34.059956 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.060033 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260808 15:05:34.060326 22962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:33253: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab.
Entry for principal kudu/127.2.180.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab.
I20260808 15:05:34.077149 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.253:33253
--webserver_interface=127.2.180.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab
--principal=kudu/127.2.180.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45379,127.2.180.253:33253,127.2.180.252:41309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:05:34.109644 22962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } attempt: 1
I20260808 15:05:34.110837 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.110320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55610)
0808 15:05:34.110529 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.110665 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 15:05:34.111893 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.111457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55614)
0808 15:05:34.111611 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.111705 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 15:05:34.112126 22962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:33253: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
I20260808 15:05:34.180400 22962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } attempt: 2
I20260808 15:05:34.181697 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.181117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55620)
0808 15:05:34.181382 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.181508 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 15:05:34.182739 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.182321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55626)
0808 15:05:34.182460 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.182561 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:05:34.182925 22962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:33253: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
I20260808 15:05:34.274212 22962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } attempt: 3
I20260808 15:05:34.275512 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.274958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55632)
0808 15:05:34.275201 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.275329 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 15:05:34.276638 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.276215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55642)
0808 15:05:34.276375 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.276475 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 15:05:34.276856 22962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:33253: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
W20260808 15:05:34.389711 22974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:34.389955 22974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:34.390044 22974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:34.396782 22974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:34.396875 22974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:34.396925 22974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:34.396979 22974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:34.397022 22974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:34.405982 22974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45379,127.2.180.253:33253,127.2.180.252:41309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab
--principal=kudu/127.2.180.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.253:33253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.180.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:34.407044 22974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:34.408238 22974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:34.419569 22985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:34.419623 22983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:34.419683 22982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:34.420094 22962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } attempt: 4
I20260808 15:05:34.420454 22974 server_base.cc:1034] running on GCE node
I20260808 15:05:34.421286 22974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:34.421356 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.420760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55652)
0808 15:05:34.421055 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.421184 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260808 15:05:34.422336 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.422012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55668)
0808 15:05:34.422130 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.422220 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 15:05:34.422535 22962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:33253: Network error: Client connection negotiation failed: client connection to 127.2.180.253:33253: connect: Connection refused (error 111)
I20260808 15:05:34.422833 22974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:34.424017 22974 hybrid_clock.cc:648] HybridClock initialized: now 1786201534423971 us; error 40 us; skew 500 ppm
I20260808 15:05:34.424330 22974 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201534, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:34.427161 22974 init.cc:377] Logged in from keytab as kudu/127.2.180.253@KRBTEST.COM (short username kudu)
I20260808 15:05:34.428471 22974 webserver.cc:460] Webserver started at http://127.2.180.253:42257/ using document root <none> and password file <none>
I20260808 15:05:34.429078 22974 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:34.429175 22974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:34.429414 22974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:05:34.432192 22974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/data/instance:
uuid: "6f1afbe2004e4718b176a145d3401090"
format_stamp: "Formatted at 2026-08-08 15:05:34 on dist-test-slave-jlzn"
server_key: "14931f48f76f79912b01d07dff4efbb4"
server_key_iv: "34824e75e153d28a75cf93d9d49046fe"
server_key_version: "encryptionkey@0"
I20260808 15:05:34.432914 22974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/wal/instance:
uuid: "6f1afbe2004e4718b176a145d3401090"
format_stamp: "Formatted at 2026-08-08 15:05:34 on dist-test-slave-jlzn"
server_key: "14931f48f76f79912b01d07dff4efbb4"
server_key_iv: "34824e75e153d28a75cf93d9d49046fe"
server_key_version: "encryptionkey@0"
I20260808 15:05:34.437469 22974 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:05:34.440521 22996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:34.441745 22974 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:05:34.441880 22974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/wal
uuid: "6f1afbe2004e4718b176a145d3401090"
format_stamp: "Formatted at 2026-08-08 15:05:34 on dist-test-slave-jlzn"
server_key: "14931f48f76f79912b01d07dff4efbb4"
server_key_iv: "34824e75e153d28a75cf93d9d49046fe"
server_key_version: "encryptionkey@0"
I20260808 15:05:34.442027 22974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:34.466396 22974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:34.469892 22974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:34.470106 22974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:34.486416 22974 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.253:33253
I20260808 15:05:34.486454 23073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.253:33253 every 8 connection(s)
I20260808 15:05:34.487596 22974 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
I20260808 15:05:34.491312 23074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:34.495108 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 22974
I20260808 15:05:34.495316 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-1/wal/instance
I20260808 15:05:34.495862 2771 external_mini_cluster.cc:1428] Setting key 3eb93562dd4553bb012bfa57d564d19e
I20260808 15:05:34.498548 23074 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201534, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:34.518115 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.501430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40188 (local address 127.2.180.254:45379)
0808 15:05:34.501634 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:34.501640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:34.502762 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 15:05:34.504978 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:34.504984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:34.505057 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:34.505144 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:34.506913 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.507584 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.509084 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.509513 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.509627 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.512782 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:34.512787 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:34.512790 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:34.512824 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:34.515262 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.515953 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.515958 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.515960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.516083 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.516485 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.516488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.516490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.516775 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:34.516911 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:34.517562 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 15:05:34.517795 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:05:34.518924 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.501824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45379 (local address 127.0.0.1:40188)
0808 15:05:34.502573 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.502618 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:34.504697 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:34.505394 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:34.505403 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:34.506044 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:34.506746 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.506766 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.507736 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.507739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.508943 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.508955 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.509743 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.509752 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.509938 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.510554 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:34.510576 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:34.512574 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:34.515387 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.515395 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.515409 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.515838 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.516183 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.516188 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.516192 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.516372 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.516894 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:34.516902 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:34.517201 (+ 299us) client_negotiation.cc:769] Sending connection context
0808 15:05:34.517423 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 15:05:34.517884 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":295,"threads_started":1}
WARNING: no policy specified for kudu/127.2.180.252@KRBTEST.COM; defaulting to no policy
I20260808 15:05:34.520445 23074 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.252@KRBTEST.COM" created.
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201534, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:05:34.534857 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.521150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55682)
0808 15:05:34.521346 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.521364 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:34.521470 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:34.522460 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:34.522463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:34.522916 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:34.523078 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.523088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.524427 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.524432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.525611 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.525626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.526355 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.526360 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.526552 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.527066 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:34.527087 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:34.529310 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:34.532729 (+ 3419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.532733 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.532735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.533080 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.533456 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.533459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.533461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.533587 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.534202 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:34.534205 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:34.534329 (+ 124us) client_negotiation.cc:769] Sending connection context
0808 15:05:34.534430 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 15:05:34.534624 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 15:05:34.535344 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.521362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55682 (local address 127.2.180.253:33253)
0808 15:05:34.522018 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:34.522026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:34.522057 (+ 31us) server_negotiation.cc:407] Connection header received
0808 15:05:34.522099 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:34.522106 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:34.522315 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:34.522460 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:34.523229 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.524297 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.525733 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.526215 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.526301 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.529502 (+ 3201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:34.529512 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:34.529523 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:34.529551 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:34.532606 (+ 3055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.533204 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.533212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.533215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.533348 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.533699 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.533704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.533706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.534094 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:34.534295 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:34.534623 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 15:05:34.534872 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":386,"threads_started":1}
I20260808 15:05:34.537437 23074 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:05:34.538687 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.538252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38566)
0808 15:05:34.538410 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.538507 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 15:05:34.538934 22998 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
I20260808 15:05:34.540410 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.540108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38580)
0808 15:05:34.540243 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.540317 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:05:34.540616 23074 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:41309: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab.
Entry for principal kudu/127.2.180.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab.
I20260808 15:05:34.550992 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.252:41309
--webserver_interface=127.2.180.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab
--principal=kudu/127.2.180.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:37611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45379,127.2.180.253:33253,127.2.180.252:41309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:05:34.589908 23074 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } attempt: 1
I20260808 15:05:34.590922 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.590502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38596)
0808 15:05:34.590683 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.590782 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 15:05:34.591959 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.591620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38602)
0808 15:05:34.591736 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.591849 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 15:05:34.592199 23074 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:41309: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
I20260808 15:05:34.660441 23074 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } attempt: 2
I20260808 15:05:34.661490 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.661072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38604)
0808 15:05:34.661257 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.661342 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 15:05:34.662441 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.662096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38616)
0808 15:05:34.662236 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.662305 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:05:34.662640 23074 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:41309: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
I20260808 15:05:34.726711 22962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } attempt: 5
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201533, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:05:34.741991 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.727467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55694)
0808 15:05:34.727713 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.727737 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:34.727899 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:34.729410 (+ 1511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:34.729414 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:34.729777 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:34.729964 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.729973 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.731323 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.731327 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.732233 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.732248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.733218 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.733223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.733357 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.733931 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:34.733949 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:34.735998 (+ 2049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:34.739739 (+ 3741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.739743 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.739746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.740124 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.740507 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.740510 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.740513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.740639 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.741265 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:34.741271 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:34.741415 (+ 144us) client_negotiation.cc:769] Sending connection context
0808 15:05:34.741522 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 15:05:34.741742 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 15:05:34.742208 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.728858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55694 (local address 127.2.180.253:33253)
0808 15:05:34.729051 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:34.729057 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:34.729078 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:05:34.729139 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:34.729144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:34.729242 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:34.729347 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:34.730117 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.731207 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.732387 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.733072 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.733214 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.736196 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:34.736202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:34.736207 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:34.736253 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:34.739603 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.740259 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.740266 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.740269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.740418 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.740742 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.740747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.740750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.741135 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:34.741275 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:34.741677 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 15:05:34.741909 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:05:34.743168 22962 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:05:34.744428 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.743989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38626)
0808 15:05:34.744164 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.744259 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 15:05:34.745630 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.745226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38636)
0808 15:05:34.745372 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.745460 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:05:34.745836 22962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:41309: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
I20260808 15:05:34.753881 23074 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } attempt: 3
I20260808 15:05:34.754735 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.754345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38644)
0808 15:05:34.754516 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.754589 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 15:05:34.755726 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.755333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38650)
0808 15:05:34.755538 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.755612 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
W20260808 15:05:34.755966 23074 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:41309: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
I20260808 15:05:34.797066 22962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } attempt: 1
I20260808 15:05:34.798175 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.797662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38662)
0808 15:05:34.797868 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.797984 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 15:05:34.799211 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.798848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38670)
0808 15:05:34.798984 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.799091 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:05:34.799418 22962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:41309: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
I20260808 15:05:34.867627 22962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } attempt: 2
I20260808 15:05:34.868875 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.868331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38674)
0808 15:05:34.868574 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.868687 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 15:05:34.870007 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.869600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38682)
0808 15:05:34.869786 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.869893 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 15:05:34.870412 22962 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:41309: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
W20260808 15:05:34.870327 23084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:34.870680 23084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:34.870821 23084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:34.878129 23084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:34.878230 23084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:34.878304 23084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:34.878379 23084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:34.878443 23084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:34.887682 23084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:37611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45379,127.2.180.253:33253,127.2.180.252:41309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab
--principal=kudu/127.2.180.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.252:41309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.180.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:34.888921 23084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:34.890190 23084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:05:34.899112 23074 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } attempt: 4
I20260808 15:05:34.900192 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.899663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38688)
0808 15:05:34.899899 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.899998 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 15:05:34.901386 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.900975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38694)
0808 15:05:34.901186 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.901266 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260808 15:05:34.901288 23089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:34.901319 23090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:34.901640 23074 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:41309: Network error: Client connection negotiation failed: client connection to 127.2.180.252:41309: connect: Connection refused (error 111)
W20260808 15:05:34.901736 23092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:34.902387 23084 server_base.cc:1034] running on GCE node
I20260808 15:05:34.903096 23084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:34.904345 23084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:34.905542 23084 hybrid_clock.cc:648] HybridClock initialized: now 1786201534905493 us; error 38 us; skew 500 ppm
I20260808 15:05:34.905850 23084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201534, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:34.909178 23084 init.cc:377] Logged in from keytab as kudu/127.2.180.252@KRBTEST.COM (short username kudu)
I20260808 15:05:34.910457 23084 webserver.cc:460] Webserver started at http://127.2.180.252:43647/ using document root <none> and password file <none>
I20260808 15:05:34.911039 23084 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:34.911151 23084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:34.911392 23084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:05:34.914143 23084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/data/instance:
uuid: "5f1bd24519b348bba18daf828e6cf676"
format_stamp: "Formatted at 2026-08-08 15:05:34 on dist-test-slave-jlzn"
server_key: "33fccdb0cd666ad854f21e46b8dc7760"
server_key_iv: "186e86c57a01d6c599df56a88cb90122"
server_key_version: "encryptionkey@0"
I20260808 15:05:34.914865 23084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/wal/instance:
uuid: "5f1bd24519b348bba18daf828e6cf676"
format_stamp: "Formatted at 2026-08-08 15:05:34 on dist-test-slave-jlzn"
server_key: "33fccdb0cd666ad854f21e46b8dc7760"
server_key_iv: "186e86c57a01d6c599df56a88cb90122"
server_key_version: "encryptionkey@0"
I20260808 15:05:34.919577 23084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260808 15:05:34.922756 23102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:34.923964 23084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260808 15:05:34.924111 23084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/wal
uuid: "5f1bd24519b348bba18daf828e6cf676"
format_stamp: "Formatted at 2026-08-08 15:05:34 on dist-test-slave-jlzn"
server_key: "33fccdb0cd666ad854f21e46b8dc7760"
server_key_iv: "186e86c57a01d6c599df56a88cb90122"
server_key_version: "encryptionkey@0"
I20260808 15:05:34.924263 23084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:34.942662 23084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:34.946498 23084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:34.946731 23084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:34.963008 23084 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.252:41309
I20260808 15:05:34.963052 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.252:41309 every 8 connection(s)
I20260808 15:05:34.964246 23084 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
I20260808 15:05:34.967612 23170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:34.970335 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 23084
I20260808 15:05:34.970548 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786201338129629-2771-0/minicluster-data/master-2/wal/instance
I20260808 15:05:34.971092 2771 external_mini_cluster.cc:1428] Setting key 19d6e79ae74c40f27ed8346c92f65d4a
I20260808 15:05:34.974903 23170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:05:34.976619 22962 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } attempt: 3
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201533, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
Aug 08 15:05:34 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201534, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:34.993557 22968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.977186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38716)
0808 15:05:34.977391 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.977411 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:34.977525 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:34.978792 (+ 1267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:34.978797 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:34.979257 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:34.979516 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.979530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.980884 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.980889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.982189 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.982200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.983038 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.983044 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.983206 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.983774 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:34.983823 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:34.986357 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:34.990415 (+ 4058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.990421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.990425 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.990917 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.991376 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.991381 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.991385 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.991559 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.992507 (+ 948us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:34.992512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:34.992691 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 15:05:34.992909 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 15:05:34.993150 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71,"mutex_wait_us":50}
I20260808 15:05:34.994521 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.977776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40190 (local address 127.2.180.254:45379)
0808 15:05:34.977953 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:34.977959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:34.978871 (+ 912us) server_negotiation.cc:407] Connection header received
0808 15:05:34.979134 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:34.979143 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:34.979215 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:34.979319 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:34.980733 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.981791 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.983789 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.984506 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.984656 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.988514 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:34.988520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:34.988523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:34.988771 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:34.991615 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.992299 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.992304 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.992308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.992436 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.992884 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.992889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.992892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.993194 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:34.993324 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:34.993947 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 15:05:34.994152 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":62}
I20260808 15:05:34.994952 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.972871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.2.180.252:41309)
0808 15:05:34.973569 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:34.973581 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:34.973620 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:05:34.975222 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:34.975243 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:34.975599 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:34.975955 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:34.978361 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.980303 (+ 1942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.981823 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.982904 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.983055 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.987955 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:34.987962 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:34.987966 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:34.988005 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:34.990975 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.991721 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.991725 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.991728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.991882 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.992343 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.992347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.992350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.992683 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:34.992900 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:34.993359 (+ 459us) server_negotiation.cc:299] Negotiation successful
0808 15:05:34.993917 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":605,"server-negotiator.queue_time_us":508,"thread_start_us":322,"threads_started":1}
I20260808 15:05:34.994995 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.977375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38716 (local address 127.2.180.252:41309)
0808 15:05:34.978422 (+ 1047us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:34.978431 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:34.978453 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:05:34.978510 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:34.978514 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:34.978590 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:34.979408 (+ 818us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:34.979682 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.980730 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.982314 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.982900 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.983053 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.986549 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:34.986557 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:34.986569 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:34.986601 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:34.990222 (+ 3621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.991042 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.991050 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.991059 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.991254 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:34.991684 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:34.991689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:34.991692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:34.992380 (+ 688us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:34.992562 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:34.993963 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0808 15:05:34.994147 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":889,"thread_start_us":732,"threads_started":1}
I20260808 15:05:34.995461 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.977690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45379 (local address 127.0.0.1:40190)
0808 15:05:34.978744 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:34.978792 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:34.978991 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:34.979395 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:34.979402 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:34.980239 (+ 837us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:34.980591 (+ 352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.980600 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.981979 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.981984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.983606 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:34.983622 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:34.984739 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:34.984748 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:34.984915 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:34.985451 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:34.985472 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:34.988117 (+ 2645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:34.991744 (+ 3627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.991753 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.991758 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.992190 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.992569 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:34.992573 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:34.992577 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:34.992746 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:34.993419 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:34.993426 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:34.993619 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 15:05:34.994740 (+ 1121us) client_negotiation.cc:240] Negotiation successful
0808 15:05:34.995046 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":36,"spinlock_wait_cycles":34688,"thread_start_us":673,"threads_started":1}
I20260808 15:05:34.997179 23170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:05:35.001109 22962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b: Bootstrap starting.
I20260808 15:05:35.005239 22962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:35.006774 22962 log.cc:824] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:05:35 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201534, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
Aug 08 15:05:35 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:35.010052 22962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b: No bootstrap required, opened a new log
I20260808 15:05:35.014091 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.998021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:33253 (local address 127.0.0.1:55696)
0808 15:05:35.000140 (+ 2119us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:35.000158 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:35.000272 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:35.000629 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:35.000634 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:35.001026 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:35.001199 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:35.001211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.002293 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.002299 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:35.003318 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:35.003332 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.004446 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.004451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:35.004695 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.005247 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:35.005270 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:35.007673 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:35.011454 (+ 3781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:35.011459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:35.011463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:35.012007 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:35.012426 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:35.012431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:35.012435 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:35.012600 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:35.013216 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:35.013221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:35.013367 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 15:05:35.013507 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 15:05:35.013765 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2009}
I20260808 15:05:35.014235 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.998416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55696 (local address 127.2.180.253:33253)
0808 15:05:34.998568 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:34.998572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:35.000233 (+ 1661us) server_negotiation.cc:407] Connection header received
0808 15:05:35.000403 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:35.000408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:35.000486 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:35.000584 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:35.001413 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.002154 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.003469 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.004122 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.004250 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.007880 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:35.007887 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:35.007892 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:35.007924 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:35.011295 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.012129 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.012135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.012138 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.012319 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.012739 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.012744 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.012748 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.013086 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:35.013235 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:35.013623 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 15:05:35.013832 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:05:35.014775 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:34.999962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40196 (local address 127.2.180.254:45379)
0808 15:05:35.000096 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:35.000100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:35.000113 (+ 13us) server_negotiation.cc:407] Connection header received
0808 15:05:35.000155 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:35.000158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:35.000209 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:35.000296 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:35.001655 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.002749 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.004324 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.004755 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.004862 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.008959 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:35.008964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:35.008967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:35.008996 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:35.011567 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.012291 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.012297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.012301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.012445 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.012842 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.012847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.012851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.013154 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:35.013408 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:35.013949 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 15:05:35.014493 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:05:35.015322 23170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:05:35.020469 22962 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
I20260808 15:05:35.020839 22962 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:35.020964 22962 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c7ee1af76e64a3aa78cc65eb4b2819b, State: Initialized, Role: FOLLOWER
I20260808 15:05:35.021621 22962 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
Aug 08 15:05:35 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201534, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
Aug 08 15:05:35 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:05:35.033200 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:35.016219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.2.180.252:41309)
0808 15:05:35.016377 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:35.016382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:35.016491 (+ 109us) server_negotiation.cc:407] Connection header received
0808 15:05:35.016613 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:35.016617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:35.016679 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:35.016774 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:35.017566 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.018571 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.020205 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.020783 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.020880 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.025505 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:35.025512 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:35.025517 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:35.025551 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:35.029426 (+ 3875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.031088 (+ 1662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.031094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.031097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.031258 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.031644 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.031649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.031652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.032025 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:35.032152 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:35.032607 (+ 455us) server_negotiation.cc:299] Negotiation successful
0808 15:05:35.032829 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:05:35.034206 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:35.016174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38722)
0808 15:05:35.016410 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:35.016427 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:35.016524 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:35.016804 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:35.016809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:35.017246 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:35.017425 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:35.017436 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.018715 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.018720 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:35.020053 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:35.020079 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.021316 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.021322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:35.021495 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.022156 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:35.022180 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:35.025330 (+ 3150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:35.030444 (+ 5114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:35.030449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:35.030454 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:35.030957 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:35.031380 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:35.031385 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:35.031388 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:35.031544 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:35.032228 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:35.032233 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:35.032387 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 15:05:35.033620 (+ 1233us) client_negotiation.cc:240] Negotiation successful
0808 15:05:35.033892 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260808 15:05:35.033950 23178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.034330 23178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:05:35.035243 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:35.017246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55712 (local address 127.2.180.253:33253)
0808 15:05:35.017388 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:35.017391 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:35.017403 (+ 12us) server_negotiation.cc:407] Connection header received
0808 15:05:35.017437 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:35.017440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:35.017485 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:35.017561 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:35.019066 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.020142 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.021979 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.022895 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.022996 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.028123 (+ 5127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:35.028129 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:35.028134 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:35.028179 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:35.032085 (+ 3906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.032982 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.032988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.032992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.033147 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.033606 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.033611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.033614 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.033941 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:35.034187 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:35.034736 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 15:05:35.034936 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:05:35.035243 22962 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:35.040284 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676: Bootstrap starting.
I20260808 15:05:35.043102 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:05:35.044677 23170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:35.045857 23170 log.cc:824] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676: Log is configured to *not* fsync() on all Append() calls
W20260808 15:05:35.051437 23193 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:05:35.053988 23193 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:05:35.044056 22867 master_runner.cc:186] Error getting master registration for 127.2.180.253:33253: OK, instance_id { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" instance_seqno: 1786201534481862 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:05:35.054816 23170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676: No bootstrap required, opened a new log
I20260808 15:05:35.054883 22867 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:05:35.065541 23170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
I20260808 15:05:35.065971 23170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:35.066107 23170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f1bd24519b348bba18daf828e6cf676, State: Initialized, Role: FOLLOWER
I20260808 15:05:35.066800 23170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
I20260808 15:05:35.075915 23207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.076323 23207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:05:35.076833 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:35.058306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40212 (local address 127.2.180.254:45379)
0808 15:05:35.058670 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:35.058675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:35.058693 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:05:35.058745 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:35.058750 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:35.058818 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:35.058919 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:35.061145 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.062288 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.064548 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.065256 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.065359 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.069498 (+ 4139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:35.069506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:35.069511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:35.069566 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:35.073170 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.074146 (+ 976us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.074151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.074155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.074301 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.074800 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.074806 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.074810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.075208 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:35.075463 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:35.076077 (+ 614us) server_negotiation.cc:299] Negotiation successful
0808 15:05:35.076470 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233}
I20260808 15:05:35.078166 23170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:35.086964 23078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:35.064977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.2.180.253:33253)
0808 15:05:35.065135 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:35.065140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:35.069943 (+ 4803us) server_negotiation.cc:407] Connection header received
0808 15:05:35.070361 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:35.070367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:35.070444 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:35.070514 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:35.072271 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.073354 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.075558 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.076175 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.076276 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.079592 (+ 3316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:35.079599 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:35.079604 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:35.079643 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:35.083297 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.084218 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.084225 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.084229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.084392 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.085131 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.085137 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.085141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.085505 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:35.085751 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:35.086336 (+ 585us) server_negotiation.cc:299] Negotiation successful
0808 15:05:35.086586 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:05:35.090724 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:35.058478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.2.180.252:41309)
0808 15:05:35.058679 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:35.058685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:35.067841 (+ 9156us) server_negotiation.cc:407] Connection header received
0808 15:05:35.068739 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:35.068744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:35.068818 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:35.068883 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:35.072328 (+ 3445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.073354 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.079946 (+ 6592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.080677 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.080802 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.083795 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:35.083824 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:35.083829 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:35.083866 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:35.087247 (+ 3381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.088366 (+ 1119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.088372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.088374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.088512 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.089025 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.089031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.089035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.089453 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:35.089727 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:35.090115 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 15:05:35.090349 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:05:35.101430 23178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:05:35.101764 23178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
I20260808 15:05:35.103502 23178 leader_election.cc:288] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f1afbe2004e4718b176a145d3401090 (127.2.180.253:33253), 5f1bd24519b348bba18daf828e6cf676 (127.2.180.252:41309)
I20260808 15:05:35.104341 23137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f1bd24519b348bba18daf828e6cf676" is_pre_election: true
I20260808 15:05:35.104946 23137 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7ee1af76e64a3aa78cc65eb4b2819b in term 0.
I20260808 15:05:35.105630 22893 leader_election.cc:302] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c7ee1af76e64a3aa78cc65eb4b2819b, 5f1bd24519b348bba18daf828e6cf676; no voters:
I20260808 15:05:35.106142 23178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:05:35.106287 23178 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:05:35.106382 23178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:35.107589 23223 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 15:05:35.109663 23178 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
I20260808 15:05:35.111287 23137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f1bd24519b348bba18daf828e6cf676"
I20260808 15:05:35.111493 23137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:35.111954 23178 leader_election.cc:288] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [CANDIDATE]: Term 1 election: Requested vote from peers 6f1afbe2004e4718b176a145d3401090 (127.2.180.253:33253), 5f1bd24519b348bba18daf828e6cf676 (127.2.180.252:41309)
I20260808 15:05:35.114609 23137 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c7ee1af76e64a3aa78cc65eb4b2819b in term 1.
I20260808 15:05:35.114979 22893 leader_election.cc:302] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c7ee1af76e64a3aa78cc65eb4b2819b, 5f1bd24519b348bba18daf828e6cf676; no voters:
I20260808 15:05:35.115278 23178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:05:35.115911 23178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [term 1 LEADER]: Becoming Leader. State: Replica: 2c7ee1af76e64a3aa78cc65eb4b2819b, State: Running, Role: LEADER
I20260808 15:05:35.116452 23178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
I20260808 15:05:35.119280 23225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c7ee1af76e64a3aa78cc65eb4b2819b. Latest consensus state: current_term: 1 leader_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.119608 23225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:35.114809 23084 master_runner.cc:186] Error getting master registration for 127.2.180.253:33253: OK, instance_id { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" instance_seqno: 1786201534481862 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:05:35.128372 23084 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:05:35.132519 23228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:05:35.134696 23228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:05:35.143134 23137 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [term 1 FOLLOWER]: Refusing update from remote peer 2c7ee1af76e64a3aa78cc65eb4b2819b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:05:35.144021 23225 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:05:35.150884 23207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c7ee1af76e64a3aa78cc65eb4b2819b. Latest consensus state: current_term: 1 leader_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.151252 23207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:05:35.154500 23229 mvcc.cc:205] Tried to move back new op lower bound from 7316281487937077248 to 7316281487841226752. Current Snapshot: MvccSnapshot[applied={T|T < 7316281487937077248}]
I20260808 15:05:35.155665 23225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.155941 23225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:35.156255 23230 mvcc.cc:205] Tried to move back new op lower bound from 7316281487937077248 to 7316281487841226752. Current Snapshot: MvccSnapshot[applied={T|T < 7316281487937077248}]
I20260808 15:05:35.156751 23207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.156965 23207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:05:35.157805 23228 catalog_manager.cc:1347] Generated new cluster ID: 7d2e7dc64b384dbf8f6115a9aefc779b
I20260808 15:05:35.157907 23228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:05:35.173306 23228 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:05:35.175037 23228 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:05:35.191293 23228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b: Generated new TSK 0
I20260808 15:05:35.192291 23228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:05:35.200868 23074 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } attempt: 5
Aug 08 15:05:35 dist-test-slave-jlzn krb5kdc[22846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201534, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
I20260808 15:05:35.218194 23076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:35.201410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:41309 (local address 127.0.0.1:38746)
0808 15:05:35.201745 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:35.201766 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:35.201870 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:35.202507 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:35.202512 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:35.203058 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:35.203271 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:35.203285 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.204826 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.204832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:35.206275 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:35.206290 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.207230 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.207237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:35.207477 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.208091 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:35.208115 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:35.211062 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:35.215112 (+ 4050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:35.215118 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:35.215121 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:35.215693 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:35.216209 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:35.216215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:35.216219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:35.216386 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:35.217153 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:35.217159 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:35.217318 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 15:05:35.217473 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 15:05:35.217769 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260808 15:05:35.219317 23173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:35.202052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38746 (local address 127.2.180.252:41309)
0808 15:05:35.202218 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:35.202224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:35.202240 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:05:35.202289 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:35.202293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:35.202363 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:35.202462 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:35.203578 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.204694 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.206410 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:35.207073 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:35.207199 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:35.211336 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:35.211343 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:35.211349 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:35.211390 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:35.214878 (+ 3488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.215885 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.215892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.215895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.216047 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:35.216525 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:35.216544 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:35.216548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:35.216915 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:35.217046 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:35.218730 (+ 1684us) server_negotiation.cc:299] Negotiation successful
0808 15:05:35.218979 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:05:35.225503 23074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090: Bootstrap starting.
I20260808 15:05:35.229647 23074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:35.230888 23074 log.cc:824] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:35.234272 23074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090: No bootstrap required, opened a new log
I20260808 15:05:35.244863 23074 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
I20260808 15:05:35.245289 23074 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:35.245414 23074 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f1afbe2004e4718b176a145d3401090, State: Initialized, Role: FOLLOWER
I20260808 15:05:35.246114 23074 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } }
I20260808 15:05:35.247020 23241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.247483 23241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:05:35.248257 23074 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:35.248310 23034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f1afbe2004e4718b176a145d3401090"
I20260808 15:05:35.248703 23033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:35.252805 23033 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [term 1 FOLLOWER]: Refusing update from remote peer 2c7ee1af76e64a3aa78cc65eb4b2819b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:05:35.253299 23035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f1afbe2004e4718b176a145d3401090" is_pre_election: true
I20260808 15:05:35.253687 23035 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c7ee1af76e64a3aa78cc65eb4b2819b in term 1.
I20260808 15:05:35.253729 23178 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:05:35.276026 23241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c7ee1af76e64a3aa78cc65eb4b2819b. Latest consensus state: current_term: 1 leader_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.276341 23241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:05:35.276512 23261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.276851 23261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:05:35.276882 23272 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 15:05:35.637704 23178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c7ee1af76e64a3aa78cc65eb4b2819b" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45379 } } peers { permanent_uuid: "6f1afbe2004e4718b176a145d3401090" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 33253 } } peers { permanent_uuid: "5f1bd24519b348bba18daf828e6cf676" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 41309 } } }
I20260808 15:05:35.638028 23178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c7ee1af76e64a3aa78cc65eb4b2819b [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:36.111879 23223 catalog_manager.cc:1259] Loaded cluster ID: 7d2e7dc64b384dbf8f6115a9aefc779b
I20260808 15:05:36.111984 23223 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676: loading cluster ID for follower catalog manager: success
I20260808 15:05:36.114396 23223 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676: acquiring CA information for follower catalog manager: success
I20260808 15:05:36.116197 23223 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5f1bd24519b348bba18daf828e6cf676: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:05:36.281987 23272 catalog_manager.cc:1259] Loaded cluster ID: 7d2e7dc64b384dbf8f6115a9aefc779b
I20260808 15:05:36.282111 23272 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090: loading cluster ID for follower catalog manager: success
I20260808 15:05:36.285648 23272 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090: acquiring CA information for follower catalog manager: success
I20260808 15:05:36.288018 23272 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f1afbe2004e4718b176a145d3401090: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:05:36.298779 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 22867
I20260808 15:05:36.299999 22954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:36.484964 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 22867
I20260808 15:05:36.510857 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 22974
I20260808 15:05:36.512099 23067 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:36.616122 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 22974
I20260808 15:05:36.641022 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 23084
I20260808 15:05:36.642112 23164 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:36.742640 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 23084
2026-08-08T15:05:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4523 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:05:36 dist-test-slave-jlzn krb5kdc[23287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:05:36 dist-test-slave-jlzn krb5kdc[23287](info): set up 2 sockets
Aug 08 15:05:36 dist-test-slave-jlzn krb5kdc[23287](info): commencing operation
krb5kdc: starting...
W20260808 15:05:37.911900 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.003s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:05:38 dist-test-slave-jlzn krb5kdc[23287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:05:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:05:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:05:38.125595 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46267
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:46267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:38.443682 23304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:38.443974 23304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:38.444113 23304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:38.451030 23304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:38.451133 23304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:38.451217 23304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:38.451296 23304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:38.451361 23304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:38.460991 23304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:46267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:38.462136 23304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:38.463317 23304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:38.474979 23314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:38.475013 23316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:38.474948 23320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:38.475636 23304 server_base.cc:1034] running on GCE node
I20260808 15:05:38.476361 23304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:38.477715 23304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:38.478915 23304 hybrid_clock.cc:648] HybridClock initialized: now 1786201538478864 us; error 39 us; skew 500 ppm
I20260808 15:05:38.479229 23304 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:38 dist-test-slave-jlzn krb5kdc[23287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201538, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:38.486408 23304 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:05:38.487767 23304 webserver.cc:460] Webserver started at http://127.2.180.254:45169/ using document root <none> and password file <none>
I20260808 15:05:38.488406 23304 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:38.488519 23304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:38.488828 23304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:05:38.491611 23304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "e1f0a0c199ce4a0a9d1f33386be263d9"
format_stamp: "Formatted at 2026-08-08 15:05:38 on dist-test-slave-jlzn"
server_key: "3a70e40520886e65f98d29f890cb5b89"
server_key_iv: "bd519cb606f947c6a61fea68a3cddd31"
server_key_version: "encryptionkey@0"
I20260808 15:05:38.492364 23304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "e1f0a0c199ce4a0a9d1f33386be263d9"
format_stamp: "Formatted at 2026-08-08 15:05:38 on dist-test-slave-jlzn"
server_key: "3a70e40520886e65f98d29f890cb5b89"
server_key_iv: "bd519cb606f947c6a61fea68a3cddd31"
server_key_version: "encryptionkey@0"
I20260808 15:05:38.497174 23304 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 15:05:38.500280 23328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:38.501398 23304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260808 15:05:38.501547 23304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "e1f0a0c199ce4a0a9d1f33386be263d9"
format_stamp: "Formatted at 2026-08-08 15:05:38 on dist-test-slave-jlzn"
server_key: "3a70e40520886e65f98d29f890cb5b89"
server_key_iv: "bd519cb606f947c6a61fea68a3cddd31"
server_key_version: "encryptionkey@0"
I20260808 15:05:38.501734 23304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:38.512740 23304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:38.516579 23304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:38.516824 23304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:38.533427 23304 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:46267
I20260808 15:05:38.533455 23402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:46267 every 8 connection(s)
I20260808 15:05:38.534629 23304 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:05:38.537711 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 23304
I20260808 15:05:38.537910 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:05:38.537985 23403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:38.538609 2771 external_mini_cluster.cc:1428] Setting key 105ace2f0aa2444fd3a703d2bae171a3
I20260808 15:05:38.551084 23403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9: Bootstrap starting.
Aug 08 15:05:38 dist-test-slave-jlzn krb5kdc[23287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:38.554127 23403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:38.555109 23403 log.cc:824] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:38.557930 23403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9: No bootstrap required, opened a new log
I20260808 15:05:38.560194 23406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:38.540280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.2.180.254:46267)
0808 15:05:38.541094 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:38.541113 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:38.541168 (+ 55us) server_negotiation.cc:407] Connection header received
0808 15:05:38.542746 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:38.542773 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:38.543229 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:38.543629 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:38.544950 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:38.546792 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:38.548254 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:38.549279 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:38.549470 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:38.553489 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:38.553496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:38.553508 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:38.553540 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:38.556428 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:38.557177 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:38.557184 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:38.557189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:38.557343 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:38.557729 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:38.557732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:38.557734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:38.558218 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:38.558454 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:38.558872 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 15:05:38.559181 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":344,"threads_started":1}
I20260808 15:05:38.563606 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:05:38.566443 23403 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f0a0c199ce4a0a9d1f33386be263d9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46267 } }
I20260808 15:05:38.566768 23403 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:38.566865 23403 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1f0a0c199ce4a0a9d1f33386be263d9, State: Initialized, Role: FOLLOWER
I20260808 15:05:38.567376 23403 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f0a0c199ce4a0a9d1f33386be263d9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46267 } }
I20260808 15:05:38.567632 23403 raft_consensus.cc:388] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:05:38.567764 23403 raft_consensus.cc:482] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:05:38.567925 23403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:38.569818 23403 raft_consensus.cc:504] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f0a0c199ce4a0a9d1f33386be263d9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46267 } }
I20260808 15:05:38.570243 23403 leader_election.cc:302] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e1f0a0c199ce4a0a9d1f33386be263d9; no voters:
I20260808 15:05:38.570765 23403 leader_election.cc:288] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:05:38.570868 23410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:05:38.571491 23410 raft_consensus.cc:686] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [term 1 LEADER]: Becoming Leader. State: Replica: e1f0a0c199ce4a0a9d1f33386be263d9, State: Running, Role: LEADER
I20260808 15:05:38.572007 23410 consensus_queue.cc:237] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f0a0c199ce4a0a9d1f33386be263d9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46267 } }
I20260808 15:05:38.572628 23403 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:38.574460 23411 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e1f0a0c199ce4a0a9d1f33386be263d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f0a0c199ce4a0a9d1f33386be263d9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46267 } } }
I20260808 15:05:38.574908 23411 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:38.574484 23412 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e1f0a0c199ce4a0a9d1f33386be263d9. Latest consensus state: current_term: 1 leader_uuid: "e1f0a0c199ce4a0a9d1f33386be263d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e1f0a0c199ce4a0a9d1f33386be263d9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46267 } } }
I20260808 15:05:38.575510 23412 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:38.576436 23421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:05:38.580466 23421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:05:38.588263 23421 catalog_manager.cc:1347] Generated new cluster ID: 994f371fb2cf4dbe90452d3813a04da6
I20260808 15:05:38.588368 23421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:05:38.596789 23421 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:05:38.597895 23421 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:05:38.605870 23421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e1f0a0c199ce4a0a9d1f33386be263d9: Generated new TSK 0
I20260808 15:05:38.606550 23421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:05:38.624580 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 23304
I20260808 15:05:38.625679 23397 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:38.712363 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 23304
2026-08-08T15:05:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1966 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:05:38 dist-test-slave-jlzn krb5kdc[23445](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:05:38 dist-test-slave-jlzn krb5kdc[23445](info): set up 2 sockets
Aug 08 15:05:38 dist-test-slave-jlzn krb5kdc[23445](info): commencing operation
krb5kdc: starting...
W20260808 15:05:39.884451 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:05:40 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:05:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:05:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:05:40.097055 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45669
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:40.408237 23463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:40.408485 23463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:40.408609 23463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:40.415545 23463 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:40.415642 23463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:40.415728 23463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:40.415819 23463 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:40.415884 23463 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:40.425298 23463 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:40.426540 23463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:40.427838 23463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:40.439725 23469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:40.439641 23470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:40.439675 23474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:40.440724 23463 server_base.cc:1034] running on GCE node
I20260808 15:05:40.441439 23463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:40.442694 23463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:40.443897 23463 hybrid_clock.cc:648] HybridClock initialized: now 1786201540443845 us; error 44 us; skew 500 ppm
I20260808 15:05:40.444211 23463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:40 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201540, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:40.447067 23463 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:05:40.448410 23463 webserver.cc:460] Webserver started at http://127.2.180.254:36285/ using document root <none> and password file <none>
I20260808 15:05:40.448995 23463 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:40.449092 23463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:40.449329 23463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:05:40.452183 23463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "4b04a0a6e99cfd6146341d80ab6ec5e4"
server_key_iv: "b5a624d91a81ff3541deabcf003756ec"
server_key_version: "encryptionkey@0"
I20260808 15:05:40.452877 23463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "4b04a0a6e99cfd6146341d80ab6ec5e4"
server_key_iv: "b5a624d91a81ff3541deabcf003756ec"
server_key_version: "encryptionkey@0"
I20260808 15:05:40.457517 23463 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:05:40.460602 23483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:40.461781 23463 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:05:40.461935 23463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "4b04a0a6e99cfd6146341d80ab6ec5e4"
server_key_iv: "b5a624d91a81ff3541deabcf003756ec"
server_key_version: "encryptionkey@0"
I20260808 15:05:40.462101 23463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:40.476804 23463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:40.480551 23463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:40.480796 23463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:40.497366 23463 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:45669
I20260808 15:05:40.497406 23553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:45669 every 8 connection(s)
I20260808 15:05:40.498548 23463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:05:40.502020 23556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:40.504864 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 23463
I20260808 15:05:40.505056 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:05:40.505708 2771 external_mini_cluster.cc:1428] Setting key 612e8a8cc3b6d74b6c1e37aa8144efce
I20260808 15:05:40.513545 23556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Bootstrap starting.
I20260808 15:05:40.516780 23556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:40.517679 23556 log.cc:824] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:05:40 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:40.520601 23556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: No bootstrap required, opened a new log
I20260808 15:05:40.527029 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:40.507433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54336 (local address 127.2.180.254:45669)
0808 15:05:40.508118 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:40.508129 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:40.508169 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:05:40.509741 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:40.509763 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:40.510101 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:40.510386 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:40.512119 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:40.513334 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:40.515131 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:40.515779 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:40.515968 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:40.520151 (+ 4183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:40.520158 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:40.520171 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:40.520204 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:40.523193 (+ 2989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:40.524008 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:40.524015 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:40.524020 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:40.524195 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:40.524574 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:40.524577 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:40.524579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:40.525116 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:40.525355 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:40.525764 (+ 409us) server_negotiation.cc:299] Negotiation successful
0808 15:05:40.526078 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":308,"threads_started":1}
I20260808 15:05:40.528784 23556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:40.529192 23556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:40.529310 23556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0700fc6a2ba3417a8e4f9a1cdb902de3, State: Initialized, Role: FOLLOWER
I20260808 15:05:40.529891 23556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:40.530107 23556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:05:40.530208 23556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:05:40.530325 23556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:40.532281 23556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:40.532689 23556 leader_election.cc:302] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0700fc6a2ba3417a8e4f9a1cdb902de3; no voters:
I20260808 15:05:40.533198 23556 leader_election.cc:288] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:05:40.533308 23564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:05:40.533910 23564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 1 LEADER]: Becoming Leader. State: Replica: 0700fc6a2ba3417a8e4f9a1cdb902de3, State: Running, Role: LEADER
I20260808 15:05:40.534403 23564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:40.534919 23556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:40.537151 23565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } } }
I20260808 15:05:40.537212 23566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0700fc6a2ba3417a8e4f9a1cdb902de3. Latest consensus state: current_term: 1 leader_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } } }
I20260808 15:05:40.537497 23565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:40.537567 23566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:40.539122 23572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:05:40.543967 23572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:05:40.551211 23572 catalog_manager.cc:1347] Generated new cluster ID: 0a3fea95ee0a4a5ea0399853378587d0
I20260808 15:05:40.551313 23572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:05:40.566448 23572 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:05:40.567541 23572 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:05:40.575328 23572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Generated new TSK 0
I20260808 15:05:40.575996 23572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:05:40.635979 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45669
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:40.962248 23594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:40.962496 23594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:40.962636 23594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:40.969569 23594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:40.969676 23594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:40.969816 23594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:05:40.978875 23594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:40.979996 23594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:40.981163 23594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:40.993196 23604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:40.993265 23600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:40.993242 23602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:40.994014 23594 server_base.cc:1034] running on GCE node
I20260808 15:05:40.994555 23594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:40.995385 23594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:40.996578 23594 hybrid_clock.cc:648] HybridClock initialized: now 1786201540996521 us; error 41 us; skew 500 ppm
I20260808 15:05:40.996865 23594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:40 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201540, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:40.999758 23594 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:05:41.001056 23594 webserver.cc:460] Webserver started at http://127.2.180.193:45697/ using document root <none> and password file <none>
I20260808 15:05:41.001641 23594 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:41.001739 23594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:41.001981 23594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:05:41.004731 23594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "d3542d670e831a225b21f4e36e27e7e5"
server_key_iv: "f9e841c2d26e28b2283ca53ac18a6382"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.005419 23594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "d3542d670e831a225b21f4e36e27e7e5"
server_key_iv: "f9e841c2d26e28b2283ca53ac18a6382"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.010104 23594 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:05:41.013342 23613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:41.014556 23594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 15:05:41.014700 23594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "d3542d670e831a225b21f4e36e27e7e5"
server_key_iv: "f9e841c2d26e28b2283ca53ac18a6382"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.014868 23594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:41.036823 23594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:41.040570 23594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:41.040812 23594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:41.041706 23594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:41.042907 23594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:41.042984 23594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:41.043104 23594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:41.043157 23594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:41.072111 23594 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:38445
I20260808 15:05:41.072142 23779 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:38445 every 8 connection(s)
I20260808 15:05:41.073232 23594 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:05:41.078049 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 23594
I20260808 15:05:41.078277 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:05:41.079072 2771 external_mini_cluster.cc:1428] Setting key f97e074d24a93008710bdec9440dcdcf
Aug 08 15:05:41 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201540, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:41.093080 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:41.075443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:59217 (local address 127.2.180.254:45669)
0808 15:05:41.075963 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:41.075969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:41.076884 (+ 915us) server_negotiation.cc:407] Connection header received
0808 15:05:41.079115 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:41.079119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:41.079204 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:41.079301 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:41.080959 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:41.081688 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:41.083740 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:41.084168 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:41.084262 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:41.087890 (+ 3628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:41.087895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:41.087898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:41.087930 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:41.090305 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:41.090968 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:41.090972 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:41.090974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:41.091105 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:41.091520 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:41.091523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:41.091525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:41.091780 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:41.091928 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:41.092664 (+ 736us) server_negotiation.cc:299] Negotiation successful
0808 15:05:41.092822 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":241,"threads_started":1}
I20260808 15:05:41.094229 23785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:41.076067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:59217)
0808 15:05:41.076738 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:41.076774 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:41.078828 (+ 2054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:41.079549 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:41.079570 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:41.080228 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:41.080780 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:41.080811 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:41.081851 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:41.081856 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:41.083613 (+ 1757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:41.083628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:41.084372 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:41.084378 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:41.084649 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:41.085269 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:41.085295 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:41.087667 (+ 2372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:41.090436 (+ 2769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:41.090444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:41.090457 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:41.090853 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:41.091209 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:41.091213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:41.091217 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:41.091399 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:41.091928 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:41.091937 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:41.092319 (+ 382us) client_negotiation.cc:769] Sending connection context
0808 15:05:41.092588 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 15:05:41.093034 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":250,"threads_started":1}
I20260808 15:05:41.095580 23780 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45669
I20260808 15:05:41.095894 23780 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:41.096657 23780 heartbeater.cc:511] Master 127.2.180.254:45669 requested a full tablet report, sending...
I20260808 15:05:41.098740 23502 ts_manager.cc:194] Registered new tserver with Master: 0d1659d7ee5a4d45a51c0ae1cec677c6 (127.2.180.193:38445)
I20260808 15:05:41.100296 23502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:59217
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:05:41.132684 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45669
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:41.443125 23788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:41.443382 23788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:41.443504 23788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:41.450445 23788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:41.450552 23788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:41.450706 23788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:05:41.459735 23788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:41.460835 23788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:41.462015 23788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:41.473987 23793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:41.474215 23794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:41.474673 23797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:41.475232 23788 server_base.cc:1034] running on GCE node
I20260808 15:05:41.475759 23788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:41.476595 23788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:41.477752 23788 hybrid_clock.cc:648] HybridClock initialized: now 1786201541477713 us; error 28 us; skew 500 ppm
I20260808 15:05:41.478106 23788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:41 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:41.481842 23788 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:05:41.483099 23788 webserver.cc:460] Webserver started at http://127.2.180.194:35719/ using document root <none> and password file <none>
I20260808 15:05:41.483734 23788 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:41.483870 23788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:41.484144 23788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:05:41.486827 23788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "dcf60a19737b46148e6d83fb2d818683"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "c6ed5e386421d99824e5348dc0dc31bd"
server_key_iv: "40a8f01e729b42913155b4ceaf7abe76"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.487535 23788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "dcf60a19737b46148e6d83fb2d818683"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "c6ed5e386421d99824e5348dc0dc31bd"
server_key_iv: "40a8f01e729b42913155b4ceaf7abe76"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.492210 23788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 15:05:41.495265 23807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:41.496425 23788 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:05:41.496563 23788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "dcf60a19737b46148e6d83fb2d818683"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "c6ed5e386421d99824e5348dc0dc31bd"
server_key_iv: "40a8f01e729b42913155b4ceaf7abe76"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.496730 23788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:41.529536 23788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:41.533428 23788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:41.533685 23788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:41.534540 23788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:41.535841 23788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:41.535933 23788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:41.536058 23788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:41.536110 23788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:41.565481 23788 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:39403
I20260808 15:05:41.565521 23969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:39403 every 8 connection(s)
I20260808 15:05:41.566737 23788 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:05:41.571623 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 23788
I20260808 15:05:41.571857 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:05:41.572403 2771 external_mini_cluster.cc:1428] Setting key ecc774124e0bf3b20ecf1ea7eaf61b97
Aug 08 15:05:41 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:41.586197 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:41.569129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:50329 (local address 127.2.180.254:45669)
0808 15:05:41.569307 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:41.569312 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:41.570381 (+ 1069us) server_negotiation.cc:407] Connection header received
0808 15:05:41.572515 (+ 2134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:41.572522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:41.572607 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:41.572690 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:41.574355 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:41.575076 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:41.576595 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:41.577010 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:41.577121 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:41.580795 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:41.580800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:41.580803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:41.580834 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:41.583207 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:41.584002 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:41.584006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:41.584008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:41.584141 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:41.584596 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:41.584600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:41.584603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:41.584945 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:41.585087 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:41.585667 (+ 580us) server_negotiation.cc:299] Negotiation successful
0808 15:05:41.585889 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:05:41.586911 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:41.569582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:50329)
0808 15:05:41.570236 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:41.570272 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:41.572226 (+ 1954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:41.572950 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:41.572961 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:41.573612 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:41.574201 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:41.574222 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:41.575219 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:41.575223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:41.576459 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:41.576471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:41.577233 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:41.577241 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:41.577445 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:41.578010 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:41.578037 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:41.580575 (+ 2538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:41.583340 (+ 2765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:41.583348 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:41.583362 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:41.583902 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:41.584239 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:41.584243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:41.584246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:41.584480 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:41.585045 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:41.585050 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:41.585323 (+ 273us) client_negotiation.cc:769] Sending connection context
0808 15:05:41.585554 (+ 231us) client_negotiation.cc:240] Negotiation successful
0808 15:05:41.585891 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":253,"threads_started":1}
I20260808 15:05:41.588231 23970 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45669
I20260808 15:05:41.588500 23970 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:41.589169 23970 heartbeater.cc:511] Master 127.2.180.254:45669 requested a full tablet report, sending...
I20260808 15:05:41.590557 23502 ts_manager.cc:194] Registered new tserver with Master: dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
I20260808 15:05:41.591343 23502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:50329
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:05:41.626365 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45669
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:41.938803 23977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:41.939049 23977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:41.939190 23977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:41.946108 23977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:41.946200 23977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:41.946352 23977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:05:41.955448 23977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:41.956542 23977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:41.957741 23977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:41.970032 23985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:41.970057 23986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:41.970089 23989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:41.971092 23977 server_base.cc:1034] running on GCE node
I20260808 15:05:41.971643 23977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:41.972515 23977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:41.973711 23977 hybrid_clock.cc:648] HybridClock initialized: now 1786201541973672 us; error 30 us; skew 500 ppm
I20260808 15:05:41.973986 23977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:41 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:41.976890 23977 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:05:41.978144 23977 webserver.cc:460] Webserver started at http://127.2.180.195:41403/ using document root <none> and password file <none>
I20260808 15:05:41.978758 23977 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:41.978874 23977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:41.979151 23977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:05:41.981843 23977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "12fd35dc0b144c29beced2c53e83d920"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "d48da4044197590463700fbf3248ecf9"
server_key_iv: "ed47f3daf376d0b72c88376f22a3bc02"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.982545 23977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "12fd35dc0b144c29beced2c53e83d920"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "d48da4044197590463700fbf3248ecf9"
server_key_iv: "ed47f3daf376d0b72c88376f22a3bc02"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.987263 23977 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 15:05:41.990542 23999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:41.991765 23977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:05:41.991966 23977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "12fd35dc0b144c29beced2c53e83d920"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "d48da4044197590463700fbf3248ecf9"
server_key_iv: "ed47f3daf376d0b72c88376f22a3bc02"
server_key_version: "encryptionkey@0"
I20260808 15:05:41.992138 23977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:42.011197 23977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:42.014765 23977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:42.015031 23977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:42.015965 23977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:42.017211 23977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:42.017293 23977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:42.017409 23977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:42.017468 23977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:42.046552 23977 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:38865
I20260808 15:05:42.046612 24176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:38865 every 8 connection(s)
I20260808 15:05:42.047744 23977 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:05:42.056663 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 23977
I20260808 15:05:42.056861 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:05:42.057297 2771 external_mini_cluster.cc:1428] Setting key fea78e2e6bbd732e495a25951862c6d3
Aug 08 15:05:42 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201541, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:42.066267 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.050057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:36575 (local address 127.2.180.254:45669)
0808 15:05:42.050228 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.050232 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.051262 (+ 1030us) server_negotiation.cc:407] Connection header received
0808 15:05:42.052997 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.053002 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.053098 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.053180 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:42.054535 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.055284 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.056972 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.057431 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.057525 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.061067 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:42.061072 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:42.061075 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:42.061107 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:42.063341 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.064083 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.064087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.064089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.064220 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.064626 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.064630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.064632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.064911 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:42.065039 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.065702 (+ 663us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.065910 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:05:42.067045 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.050421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:36575)
0808 15:05:42.051092 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:42.051134 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:42.052749 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:42.053362 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:42.053370 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:42.053862 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:42.054379 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.054395 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.055392 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.055396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.056832 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.056848 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.057650 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.057658 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.057894 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.058504 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:42.058532 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:42.060867 (+ 2335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:42.063477 (+ 2610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:42.063486 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:42.063499 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:42.063958 (+ 459us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:42.064326 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:42.064331 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:42.064335 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:42.064510 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:42.065032 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:42.065039 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:42.065362 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 15:05:42.065585 (+ 223us) client_negotiation.cc:240] Negotiation successful
0808 15:05:42.066033 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":265,"threads_started":1}
I20260808 15:05:42.068243 24177 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45669
I20260808 15:05:42.068488 24177 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:42.069103 24177 heartbeater.cc:511] Master 127.2.180.254:45669 requested a full tablet report, sending...
I20260808 15:05:42.070446 23502 ts_manager.cc:194] Registered new tserver with Master: 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865)
I20260808 15:05:42.071326 23502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:36575
I20260808 15:05:42.072441 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:05:42.088824 23784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.077889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45356 (local address 127.2.180.254:45669)
0808 15:05:42.078070 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.078076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.078430 (+ 354us) server_negotiation.cc:407] Connection header received
0808 15:05:42.078572 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.078576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.078630 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.078704 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:42.080029 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.080713 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.081895 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.082274 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.082355 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.084498 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:42.084502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:42.084505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:42.084533 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:42.086670 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.087215 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.087218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.087220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.087330 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.087641 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.087644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.087646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.087962 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:42.088131 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.088406 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.088550 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:05:42.092197 23502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45356:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:05:42.101025 23502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:05:42.103688 23780 heartbeater.cc:503] Master 127.2.180.254:45669 was elected leader, sending a full tablet report...
I20260808 15:05:42.124648 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.116109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:39403 (local address 127.0.0.1:35746)
0808 15:05:42.116739 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:42.116755 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:42.116864 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:42.117781 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:42.117784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:42.117804 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:42.118064 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.118073 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.119865 (+ 1792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.119870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.121879 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.121895 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.123661 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.123667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.123908 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.123961 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 15:05:42.124096 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 15:05:42.124291 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":385,"threads_started":1}
I20260808 15:05:42.124660 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.116638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.2.180.194:39403)
0808 15:05:42.117233 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.117242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.117272 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:05:42.117374 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.117381 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.117593 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.117780 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:42.118249 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.119710 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.122076 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.123453 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.123611 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.123838 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.124151 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.124326 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":291,"threads_started":1}
I20260808 15:05:42.128785 23876 tablet_service.cc:1467] Processing CreateTablet for tablet 24cbc782082b4a2e9778da320cc5806e (DEFAULT_TABLE table=test-table [id=73b6d2a7fa644afc8610f884162dc0c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:42.130455 23876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24cbc782082b4a2e9778da320cc5806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:42.132661 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.114022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33204 (local address 127.2.180.195:38865)
0808 15:05:42.114643 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.114649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.114691 (+ 42us) server_negotiation.cc:407] Connection header received
0808 15:05:42.114764 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.114769 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.114935 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.115082 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:42.121855 (+ 6773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.123448 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.129870 (+ 6422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.131214 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.131339 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.131575 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.132167 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.132330 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":331,"threads_started":1}
I20260808 15:05:42.133308 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.113744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:38865 (local address 127.0.0.1:33204)
0808 15:05:42.114320 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:42.114365 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:42.114524 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:42.115131 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:42.115138 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:42.115167 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:42.121644 (+ 6477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.121663 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.125011 (+ 3348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.125017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.128145 (+ 3128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.128169 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.131726 (+ 3557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.131733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.131933 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.131965 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:05:42.132847 (+ 882us) client_negotiation.cc:240] Negotiation successful
0808 15:05:42.132981 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":1072,"threads_started":1}
I20260808 15:05:42.135283 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.115488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38774)
0808 15:05:42.116028 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:42.116045 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:42.116172 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:42.128885 (+ 12713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:42.128889 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:42.128912 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:42.129227 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.129240 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.130571 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.130576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.132643 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.132659 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.134575 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.134582 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.134752 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.134779 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 15:05:42.134854 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 15:05:42.134969 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":236,"threads_started":1}
I20260808 15:05:42.135438 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.115797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38774 (local address 127.2.180.193:38445)
0808 15:05:42.128460 (+ 12663us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.128466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.128488 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:05:42.128558 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.128562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.128719 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.128873 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:42.129398 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.130450 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.132840 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.134389 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.134508 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.134713 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.134916 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.135060 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12486,"thread_start_us":13217,"threads_started":1}
I20260808 15:05:42.137856 24077 tablet_service.cc:1467] Processing CreateTablet for tablet 24cbc782082b4a2e9778da320cc5806e (DEFAULT_TABLE table=test-table [id=73b6d2a7fa644afc8610f884162dc0c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:42.139307 24077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24cbc782082b4a2e9778da320cc5806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:42.138833 23688 tablet_service.cc:1467] Processing CreateTablet for tablet 24cbc782082b4a2e9778da320cc5806e (DEFAULT_TABLE table=test-table [id=73b6d2a7fa644afc8610f884162dc0c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:42.140223 23688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24cbc782082b4a2e9778da320cc5806e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:42.157505 24202 tablet_bootstrap.cc:492] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Bootstrap starting.
I20260808 15:05:42.159622 24204 tablet_bootstrap.cc:492] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Bootstrap starting.
I20260808 15:05:42.161178 24202 tablet_bootstrap.cc:654] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:42.162549 24202 log.cc:824] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:42.164314 24204 tablet_bootstrap.cc:654] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:42.164999 24206 tablet_bootstrap.cc:492] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Bootstrap starting.
I20260808 15:05:42.168290 24206 tablet_bootstrap.cc:654] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:42.169494 24204 log.cc:824] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:42.171103 24206 log.cc:824] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:42.173646 24204 tablet_bootstrap.cc:492] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: No bootstrap required, opened a new log
I20260808 15:05:42.173935 24204 ts_tablet_manager.cc:1397] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260808 15:05:42.175474 24202 tablet_bootstrap.cc:492] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: No bootstrap required, opened a new log
I20260808 15:05:42.175693 24206 tablet_bootstrap.cc:492] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: No bootstrap required, opened a new log
I20260808 15:05:42.175751 24202 ts_tablet_manager.cc:1397] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Time spent bootstrapping tablet: real 0.018s user 0.011s sys 0.004s
I20260808 15:05:42.176013 24206 ts_tablet_manager.cc:1397] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260808 15:05:42.182912 24202 raft_consensus.cc:348] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.183247 24202 raft_consensus.cc:374] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:42.183342 24202 raft_consensus.cc:729] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf60a19737b46148e6d83fb2d818683, State: Initialized, Role: FOLLOWER
I20260808 15:05:42.183919 24202 consensus_queue.cc:260] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.184121 24204 raft_consensus.cc:348] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.184582 24204 raft_consensus.cc:374] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:42.184931 23970 heartbeater.cc:503] Master 127.2.180.254:45669 was elected leader, sending a full tablet report...
I20260808 15:05:42.185376 24202 ts_tablet_manager.cc:1428] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260808 15:05:42.185921 24206 raft_consensus.cc:348] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.184712 24204 raft_consensus.cc:729] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12fd35dc0b144c29beced2c53e83d920, State: Initialized, Role: FOLLOWER
I20260808 15:05:42.186352 24206 raft_consensus.cc:374] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:42.186506 24206 raft_consensus.cc:729] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1659d7ee5a4d45a51c0ae1cec677c6, State: Initialized, Role: FOLLOWER
I20260808 15:05:42.186756 24204 consensus_queue.cc:260] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.187099 24206 consensus_queue.cc:260] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.187839 24177 heartbeater.cc:503] Master 127.2.180.254:45669 was elected leader, sending a full tablet report...
I20260808 15:05:42.188505 24204 ts_tablet_manager.cc:1428] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 15:05:42.190840 24206 ts_tablet_manager.cc:1428] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Time spent starting tablet: real 0.015s user 0.010s sys 0.004s
W20260808 15:05:42.299206 24178 tablet.cc:2369] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:05:42.318562 23971 tablet.cc:2369] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:05:42.325337 23783 tablet.cc:2369] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:05:42.436615 24215 raft_consensus.cc:482] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:05:42.436932 24215 raft_consensus.cc:504] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.438539 24215 leader_election.cc:288] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d1659d7ee5a4d45a51c0ae1cec677c6 (127.2.180.193:38445), dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
I20260808 15:05:42.444929 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.438931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:42349 (local address 127.2.180.193:38445)
0808 15:05:42.439157 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.439163 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.439183 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:05:42.439350 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.439354 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.439413 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.439483 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:42.439949 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.440997 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.442977 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.444178 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.444292 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.444343 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.444662 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.444750 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:05:42.445053 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.438838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.2.180.195:42349)
0808 15:05:42.439042 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:42.439063 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:42.439204 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:42.439529 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:42.439534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:42.439565 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:42.439785 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.439797 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.441141 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.441146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.442819 (+ 1673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.442831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.444319 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.444325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.444515 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.444549 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 15:05:42.444648 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 15:05:42.444775 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":40}
I20260808 15:05:42.445920 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.439615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:50047 (local address 127.2.180.194:39403)
0808 15:05:42.439859 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.439865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.440050 (+ 185us) server_negotiation.cc:407] Connection header received
0808 15:05:42.440277 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.440280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.440337 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.440404 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:42.440961 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.441954 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.444043 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.445148 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.445296 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.445349 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.445659 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.445751 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:05:42.446061 24220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.439277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:39403 (local address 127.2.180.195:50047)
0808 15:05:42.439966 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:42.439985 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:42.440146 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:42.440437 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:42.440442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:42.440466 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:42.440831 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.440842 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.442099 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.442102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.443901 (+ 1799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:42.443916 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.445356 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.445363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:42.445501 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.445524 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:05:42.445608 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 15:05:42.445717 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":20,"thread_start_us":400,"threads_started":1}
I20260808 15:05:42.446193 23714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24cbc782082b4a2e9778da320cc5806e" candidate_uuid: "12fd35dc0b144c29beced2c53e83d920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" is_pre_election: true
I20260808 15:05:42.446712 23714 raft_consensus.cc:2454] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12fd35dc0b144c29beced2c53e83d920 in term 0.
I20260808 15:05:42.446943 23905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24cbc782082b4a2e9778da320cc5806e" candidate_uuid: "12fd35dc0b144c29beced2c53e83d920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf60a19737b46148e6d83fb2d818683" is_pre_election: true
I20260808 15:05:42.447351 24000 leader_election.cc:302] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d1659d7ee5a4d45a51c0ae1cec677c6, 12fd35dc0b144c29beced2c53e83d920; no voters:
I20260808 15:05:42.447431 23905 raft_consensus.cc:2454] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12fd35dc0b144c29beced2c53e83d920 in term 0.
I20260808 15:05:42.447747 24215 raft_consensus.cc:2781] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:05:42.447918 24215 raft_consensus.cc:482] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:05:42.448024 24215 raft_consensus.cc:3037] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:42.450183 24215 raft_consensus.cc:504] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.450724 24215 leader_election.cc:288] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [CANDIDATE]: Term 1 election: Requested vote from peers 0d1659d7ee5a4d45a51c0ae1cec677c6 (127.2.180.193:38445), dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
I20260808 15:05:42.451134 23714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24cbc782082b4a2e9778da320cc5806e" candidate_uuid: "12fd35dc0b144c29beced2c53e83d920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6"
I20260808 15:05:42.451154 23905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24cbc782082b4a2e9778da320cc5806e" candidate_uuid: "12fd35dc0b144c29beced2c53e83d920" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dcf60a19737b46148e6d83fb2d818683"
I20260808 15:05:42.451346 23905 raft_consensus.cc:3037] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:42.451346 23714 raft_consensus.cc:3037] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:42.454444 23714 raft_consensus.cc:2454] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12fd35dc0b144c29beced2c53e83d920 in term 1.
I20260808 15:05:42.454444 23905 raft_consensus.cc:2454] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12fd35dc0b144c29beced2c53e83d920 in term 1.
I20260808 15:05:42.454810 24000 leader_election.cc:302] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d1659d7ee5a4d45a51c0ae1cec677c6, 12fd35dc0b144c29beced2c53e83d920; no voters:
I20260808 15:05:42.455070 24215 raft_consensus.cc:2781] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:05:42.455461 24215 raft_consensus.cc:686] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 1 LEADER]: Becoming Leader. State: Replica: 12fd35dc0b144c29beced2c53e83d920, State: Running, Role: LEADER
I20260808 15:05:42.455911 24215 consensus_queue.cc:237] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } }
I20260808 15:05:42.460033 23502 catalog_manager.cc:5495] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "12fd35dc0b144c29beced2c53e83d920" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } health_report { overall_health: UNKNOWN } } }
I20260808 15:05:42.488813 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.483868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.2.180.195:38865)
0808 15:05:42.484059 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.484065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.484291 (+ 226us) server_negotiation.cc:407] Connection header received
0808 15:05:42.484444 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.484448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.484525 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.484640 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:05:42.484930 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.485630 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.487061 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.487424 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.487507 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.487935 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:05:42.488400 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:05:42.488502 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.488565 (+ 63us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.488627 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:05:42.496136 23905 raft_consensus.cc:1261] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 1 FOLLOWER]: Refusing update from remote peer 12fd35dc0b144c29beced2c53e83d920: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:05:42.496160 23714 raft_consensus.cc:1261] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Refusing update from remote peer 12fd35dc0b144c29beced2c53e83d920: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:05:42.497009 24221 consensus_queue.cc:1035] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:05:42.497423 24215 consensus_queue.cc:1035] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:05:42.508625 24224 mvcc.cc:205] Tried to move back new op lower bound from 7316281518055235584 to 7316281517902663680. Current Snapshot: MvccSnapshot[applied={T|T < 7316281518055235584}]
I20260808 15:05:42.511195 24227 mvcc.cc:205] Tried to move back new op lower bound from 7316281518055235584 to 7316281517902663680. Current Snapshot: MvccSnapshot[applied={T|T < 7316281518055235584}]
I20260808 15:05:42.526481 23500 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:05:42.526757 23500 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45356:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:05:42.530231 23500 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Sending DeleteTablet for 3 replicas of tablet 24cbc782082b4a2e9778da320cc5806e
I20260808 15:05:42.531248 24077 tablet_service.cc:1514] Processing DeleteTablet for tablet 24cbc782082b4a2e9778da320cc5806e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:42 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33204
I20260808 15:05:42.531512 23876 tablet_service.cc:1514] Processing DeleteTablet for tablet 24cbc782082b4a2e9778da320cc5806e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:42 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:35746
I20260808 15:05:42.531627 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 23594
I20260808 15:05:42.532404 23688 tablet_service.cc:1514] Processing DeleteTablet for tablet 24cbc782082b4a2e9778da320cc5806e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:42 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:38774
I20260808 15:05:42.543743 24242 tablet_replica.cc:330] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: stopping tablet replica
I20260808 15:05:42.544260 24243 tablet_replica.cc:330] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: stopping tablet replica
I20260808 15:05:42.544600 24243 raft_consensus.cc:2229] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:05:42.544996 24243 raft_consensus.cc:2258] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:42.545560 24242 raft_consensus.cc:2229] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:05:42.546068 24242 raft_consensus.cc:2258] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:42.546789 24245 tablet_replica.cc:330] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: stopping tablet replica
I20260808 15:05:42.547196 24245 raft_consensus.cc:2229] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:05:42.547729 24245 raft_consensus.cc:2258] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:42.547904 24243 ts_tablet_manager.cc:1905] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:05:42.550026 24245 ts_tablet_manager.cc:1905] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:05:42.551201 24242 ts_tablet_manager.cc:1905] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:05:42.558172 24245 ts_tablet_manager.cc:1918] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:05:42.558405 24245 log.cc:1196] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/24cbc782082b4a2e9778da320cc5806e
I20260808 15:05:42.559048 24245 ts_tablet_manager.cc:1939] T 24cbc782082b4a2e9778da320cc5806e P 0d1659d7ee5a4d45a51c0ae1cec677c6: Deleting consensus metadata
I20260808 15:05:42.559065 24243 ts_tablet_manager.cc:1918] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:05:42.559231 24243 log.cc:1196] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/24cbc782082b4a2e9778da320cc5806e
I20260808 15:05:42.559723 24243 ts_tablet_manager.cc:1939] T 24cbc782082b4a2e9778da320cc5806e P dcf60a19737b46148e6d83fb2d818683: Deleting consensus metadata
I20260808 15:05:42.561354 23484 catalog_manager.cc:4841] TS dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403): tablet 24cbc782082b4a2e9778da320cc5806e (table test-table [id=73b6d2a7fa644afc8610f884162dc0c7]) successfully deleted
I20260808 15:05:42.561661 23484 catalog_manager.cc:4841] TS 0d1659d7ee5a4d45a51c0ae1cec677c6 (127.2.180.193:38445): tablet 24cbc782082b4a2e9778da320cc5806e (table test-table [id=73b6d2a7fa644afc8610f884162dc0c7]) successfully deleted
I20260808 15:05:42.564996 24242 ts_tablet_manager.cc:1918] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:05:42.565191 24242 log.cc:1196] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/24cbc782082b4a2e9778da320cc5806e
I20260808 15:05:42.565816 24242 ts_tablet_manager.cc:1939] T 24cbc782082b4a2e9778da320cc5806e P 12fd35dc0b144c29beced2c53e83d920: Deleting consensus metadata
I20260808 15:05:42.567842 23484 catalog_manager.cc:4841] TS 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865): tablet 24cbc782082b4a2e9778da320cc5806e (table test-table [id=73b6d2a7fa644afc8610f884162dc0c7]) successfully deleted
Aug 08 15:05:42 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:05:42.576352 24199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.532895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.2.180.193:38445)
0808 15:05:42.533082 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.533089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.560328 (+ 27239us) server_negotiation.cc:407] Connection header received
0808 15:05:42.560565 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.560571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.560674 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.560752 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:42.562692 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.563786 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.565683 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.566275 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.566388 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.570857 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:42.570868 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:42.570880 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:42.570915 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:42.573505 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.574173 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.574180 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.574182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.574323 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.574726 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.574731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.574735 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.575058 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:42.575453 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.575771 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.576003 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:05:42.576967 23773 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:42.662338 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 23594
I20260808 15:05:42.692804 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 23788
Aug 08 15:05:42 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:05:42.709043 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.693561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35758 (local address 127.2.180.194:39403)
0808 15:05:42.693734 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.693738 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.693787 (+ 49us) server_negotiation.cc:407] Connection header received
0808 15:05:42.693988 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.693992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.694062 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.694139 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:42.695885 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.696661 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.698498 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.698873 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.698965 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.703666 (+ 4701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:42.703681 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:42.703696 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:42.703728 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:42.706365 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.707003 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.707010 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.707012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.707136 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.707481 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.707485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.707487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.707895 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:42.708190 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.708496 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.708716 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:05:42.709628 23963 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:42.796244 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 23788
I20260808 15:05:42.823771 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 23977
Aug 08 15:05:42 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:05:42.839897 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:42.824667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33220 (local address 127.2.180.195:38865)
0808 15:05:42.824864 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:42.824869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:42.824889 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:05:42.824951 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:42.824957 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:42.825025 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:42.825110 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:42.826469 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.827286 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.828736 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:42.829180 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:42.829315 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:42.833891 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:42.833903 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:42.833918 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:42.833955 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:42.837098 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.837743 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.837751 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.837753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.837903 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:42.838287 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:42.838291 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:42.838293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:42.838660 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:42.838965 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:42.839263 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 15:05:42.839479 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:05:42.840687 24167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:42.959024 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 23977
I20260808 15:05:42.978243 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 23463
I20260808 15:05:42.979297 23549 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:43.084245 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 23463
I20260808 15:05:43.110497 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45669
--webserver_interface=127.2.180.254
--webserver_port=36285
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:43.430747 24251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:43.431005 24251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:43.431130 24251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:43.438050 24251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:43.438141 24251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:43.438213 24251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:43.438275 24251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:43.438326 24251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:43.448033 24251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=36285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:43.449131 24251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:43.450388 24251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:43.474467 24259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:43.474493 24258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:43.474540 24261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:43.475279 24251 server_base.cc:1034] running on GCE node
I20260808 15:05:43.476156 24251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:43.477566 24251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:43.478775 24251 hybrid_clock.cc:648] HybridClock initialized: now 1786201543478723 us; error 41 us; skew 500 ppm
I20260808 15:05:43.479138 24251 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:43 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:43.482389 24251 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:05:43.483970 24251 webserver.cc:460] Webserver started at http://127.2.180.254:36285/ using document root <none> and password file <none>
I20260808 15:05:43.484820 24251 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:43.484977 24251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:43.491199 24251 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:05:43.493968 24268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:43.495230 24251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:05:43.495390 24251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "4b04a0a6e99cfd6146341d80ab6ec5e4"
server_key_iv: "b5a624d91a81ff3541deabcf003756ec"
server_key_version: "encryptionkey@0"
I20260808 15:05:43.496441 24251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:43.518041 24251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:43.521713 24251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:43.521951 24251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:43.539556 24251 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:45669
I20260808 15:05:43.539628 24351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:45669 every 8 connection(s)
I20260808 15:05:43.540966 24251 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:05:43.541788 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 24251
I20260808 15:05:43.542665 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:38445
--local_ip_for_outbound_sockets=127.2.180.193
--tserver_master_addrs=127.2.180.254:45669
--webserver_port=45697
--webserver_interface=127.2.180.193
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:05:43.547690 24353 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:05:43.550915 24353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Bootstrap starting.
I20260808 15:05:43.576241 24353 log.cc:824] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:43.590677 24353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:05:43.591168 24353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Bootstrap complete.
I20260808 15:05:43.600095 24353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:43.600970 24353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0700fc6a2ba3417a8e4f9a1cdb902de3, State: Initialized, Role: FOLLOWER
I20260808 15:05:43.601586 24353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:43.601809 24353 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:05:43.601920 24353 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:05:43.602088 24353 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:05:43.605763 24353 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:43.606287 24353 leader_election.cc:302] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0700fc6a2ba3417a8e4f9a1cdb902de3; no voters:
I20260808 15:05:43.606848 24353 leader_election.cc:288] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 15:05:43.607023 24358 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:05:43.608000 24358 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 2 LEADER]: Becoming Leader. State: Replica: 0700fc6a2ba3417a8e4f9a1cdb902de3, State: Running, Role: LEADER
I20260808 15:05:43.608587 24358 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:43.608867 24353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:43.610963 24361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0700fc6a2ba3417a8e4f9a1cdb902de3. Latest consensus state: current_term: 2 leader_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } } }
I20260808 15:05:43.611354 24361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:43.611935 24359 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } } }
I20260808 15:05:43.612222 24359 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:43.619841 24369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:05:43.627702 24369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:05:43.629324 24369 catalog_manager.cc:1259] Loaded cluster ID: 0a3fea95ee0a4a5ea0399853378587d0
I20260808 15:05:43.629426 24369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:05:43.632601 24369 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 15:05:43.649484 24369 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 15:05:43.666152 24369 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:05:43.667733 24369 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Loaded TSK: 0
I20260808 15:05:43.669971 24369 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:05:43.897128 24356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:43.897394 24356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:43.897533 24356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:43.904604 24356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:43.904709 24356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:43.904850 24356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:05:43.914088 24356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:38445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=45697
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:43.915218 24356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:43.916495 24356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:43.930811 24393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:43.930837 24394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:43.931859 24396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:43.931882 24356 server_base.cc:1034] running on GCE node
I20260808 15:05:43.932493 24356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:43.933336 24356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:43.934548 24356 hybrid_clock.cc:648] HybridClock initialized: now 1786201543934471 us; error 50 us; skew 500 ppm
I20260808 15:05:43.934824 24356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:43 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:43.937896 24356 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:05:43.939230 24356 webserver.cc:460] Webserver started at http://127.2.180.193:45697/ using document root <none> and password file <none>
I20260808 15:05:43.939903 24356 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:43.940034 24356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:43.945113 24356 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:05:43.947723 24406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:43.949059 24356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:05:43.949209 24356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "d3542d670e831a225b21f4e36e27e7e5"
server_key_iv: "f9e841c2d26e28b2283ca53ac18a6382"
server_key_version: "encryptionkey@0"
I20260808 15:05:43.950101 24356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:43.963611 24356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:43.967217 24356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:43.967444 24356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:43.968348 24356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:43.969595 24356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:43.969696 24356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:43.969797 24356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:43.969868 24356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:43.999368 24356 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:38445
I20260808 15:05:43.999440 24567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:38445 every 8 connection(s)
I20260808 15:05:44.000897 24356 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:05:44.001890 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 24356
I20260808 15:05:44.002535 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:39403
--local_ip_for_outbound_sockets=127.2.180.194
--tserver_master_addrs=127.2.180.254:45669
--webserver_port=35719
--webserver_interface=127.2.180.194
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:05:44 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201543, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:44.023674 24573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.003943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:49559)
0808 15:05:44.004671 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:44.004713 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:44.006351 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:44.007643 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:44.007654 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:44.008256 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:44.008767 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.008807 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.010519 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.010523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.011723 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.011736 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.012748 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.012756 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.013029 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.013692 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:44.013719 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:44.016116 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:44.019985 (+ 3869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:44.019991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:44.020001 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:44.020382 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:44.020872 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:44.020875 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:44.020877 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:44.021078 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:44.021888 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:44.021894 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:44.022191 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 15:05:44.022394 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 15:05:44.022739 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":299,"threads_started":1}
I20260808 15:05:44.024221 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.003621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:49559 (local address 127.2.180.254:45669)
0808 15:05:44.004174 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:44.004185 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:44.004836 (+ 651us) server_negotiation.cc:407] Connection header received
0808 15:05:44.006840 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:44.006852 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:44.007157 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:44.007472 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:44.009017 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.010366 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.011885 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.012517 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.012693 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.016407 (+ 3714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:44.016416 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:44.016433 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:44.016480 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:44.019865 (+ 3385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:44.020529 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:44.020537 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:44.020544 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:44.020776 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:44.021190 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:44.021194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:44.021197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:44.021759 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:44.021982 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:44.022649 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 15:05:44.023072 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":259,"threads_started":1}
I20260808 15:05:44.026813 24568 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45669
I20260808 15:05:44.027137 24568 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:44.028031 24568 heartbeater.cc:511] Master 127.2.180.254:45669 requested a full tablet report, sending...
I20260808 15:05:44.030224 24295 ts_manager.cc:194] Registered new tserver with Master: 0d1659d7ee5a4d45a51c0ae1cec677c6 (127.2.180.193:38445)
I20260808 15:05:44.031666 24295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:49559
W20260808 15:05:44.325279 24575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:44.325539 24575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:44.325675 24575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:44.332561 24575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:44.332664 24575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:44.332804 24575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:05:44.341912 24575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:39403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=35719
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:44.343029 24575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:44.344245 24575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:44.356411 24581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:44.356432 24582 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:44.356659 24585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:44.357626 24575 server_base.cc:1034] running on GCE node
I20260808 15:05:44.358168 24575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:44.359037 24575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:44.360244 24575 hybrid_clock.cc:648] HybridClock initialized: now 1786201544360160 us; error 72 us; skew 500 ppm
I20260808 15:05:44.360534 24575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:44 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:44.363338 24575 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:05:44.364661 24575 webserver.cc:460] Webserver started at http://127.2.180.194:35719/ using document root <none> and password file <none>
I20260808 15:05:44.365260 24575 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:44.365379 24575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:44.370188 24575 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:05:44.372664 24593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:44.373814 24575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:05:44.373965 24575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "dcf60a19737b46148e6d83fb2d818683"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "c6ed5e386421d99824e5348dc0dc31bd"
server_key_iv: "40a8f01e729b42913155b4ceaf7abe76"
server_key_version: "encryptionkey@0"
I20260808 15:05:44.374862 24575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:44.403117 24575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:44.406549 24575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:44.406790 24575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:44.407635 24575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:44.408877 24575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:44.408972 24575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:44.409073 24575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:44.409147 24575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:44.438663 24575 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:39403
I20260808 15:05:44.438721 24768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:39403 every 8 connection(s)
I20260808 15:05:44.440055 24575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:05:44.449440 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 24575
I20260808 15:05:44.450161 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:38865
--local_ip_for_outbound_sockets=127.2.180.195
--tserver_master_addrs=127.2.180.254:45669
--webserver_port=41403
--webserver_interface=127.2.180.195
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:05:44 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:44.458526 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.442300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:45147 (local address 127.2.180.254:45669)
0808 15:05:44.442512 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:44.442518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:44.443610 (+ 1092us) server_negotiation.cc:407] Connection header received
0808 15:05:44.445352 (+ 1742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:44.445356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:44.445425 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:44.445508 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:44.446826 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.447506 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.448988 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.449604 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.449729 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.453042 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:44.453048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:44.453052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:44.453084 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:44.455547 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:44.456258 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:44.456262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:44.456265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:44.456384 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:44.456798 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:44.456801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:44.456803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:44.457078 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:44.457217 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:44.457958 (+ 741us) server_negotiation.cc:299] Negotiation successful
0808 15:05:44.458206 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:05:44.459304 24771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.442784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:45147)
0808 15:05:44.443449 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:44.443484 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:44.445125 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:44.445711 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:44.445718 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:44.446176 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:44.446698 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.446715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.447636 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.447639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.448880 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.448893 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.449796 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.449801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.449979 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.450624 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:44.450645 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:44.452780 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:44.455674 (+ 2894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:44.455683 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:44.455697 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:44.456137 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:44.456487 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:44.456493 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:44.456497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:44.456681 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:44.457201 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:44.457209 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:44.457597 (+ 388us) client_negotiation.cc:769] Sending connection context
0808 15:05:44.457815 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 15:05:44.458258 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":267,"threads_started":1}
I20260808 15:05:44.460644 24769 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45669
I20260808 15:05:44.460955 24769 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:44.461714 24769 heartbeater.cc:511] Master 127.2.180.254:45669 requested a full tablet report, sending...
I20260808 15:05:44.463193 24295 ts_manager.cc:194] Registered new tserver with Master: dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
I20260808 15:05:44.464280 24295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:45147
W20260808 15:05:44.766139 24772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:44.766376 24772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:44.766476 24772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:44.773278 24772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:44.773368 24772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:44.773479 24772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:05:44.782635 24772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:38865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=41403
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:44.783677 24772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:44.784909 24772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:44.796787 24782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:44.796746 24784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:44.796836 24781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:44.797508 24772 server_base.cc:1034] running on GCE node
I20260808 15:05:44.798059 24772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:44.798882 24772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:44.800071 24772 hybrid_clock.cc:648] HybridClock initialized: now 1786201544800019 us; error 45 us; skew 500 ppm
I20260808 15:05:44.800364 24772 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:44 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:44.803231 24772 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:05:44.804514 24772 webserver.cc:460] Webserver started at http://127.2.180.195:41403/ using document root <none> and password file <none>
I20260808 15:05:44.805153 24772 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:44.805279 24772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:44.810163 24772 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:05:44.812618 24796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:44.813759 24772 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:05:44.813905 24772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "12fd35dc0b144c29beced2c53e83d920"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "d48da4044197590463700fbf3248ecf9"
server_key_iv: "ed47f3daf376d0b72c88376f22a3bc02"
server_key_version: "encryptionkey@0"
I20260808 15:05:44.814777 24772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:44.839787 24772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:44.843271 24772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:44.843506 24772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:44.844409 24772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:44.845669 24772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:44.845762 24772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:44.845861 24772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:44.845928 24772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:44.882038 24772 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:38865
I20260808 15:05:44.882102 24961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:38865 every 8 connection(s)
I20260808 15:05:44.883354 24772 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:05:44.886164 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 24772
Aug 08 15:05:44 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201544, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:05:44.903244 24572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.885800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:40691 (local address 127.2.180.254:45669)
0808 15:05:44.885998 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:44.886004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:44.887230 (+ 1226us) server_negotiation.cc:407] Connection header received
0808 15:05:44.889194 (+ 1964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:44.889199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:44.889282 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:44.889368 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:44.890830 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.891598 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.893610 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.894035 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.894154 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.897628 (+ 3474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:44.897632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:44.897635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:44.897662 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:44.900189 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:44.900955 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:44.900961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:44.900964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:44.901162 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:44.901564 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:44.901570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:44.901574 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:44.901910 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:44.902077 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:44.902672 (+ 595us) server_negotiation.cc:299] Negotiation successful
0808 15:05:44.902930 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 15:05:44.904021 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.886307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:40691)
0808 15:05:44.887070 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:44.887115 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:44.888935 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:44.889630 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:44.889639 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:44.890128 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:44.890662 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.890680 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.891745 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.891751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.893475 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.893488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.894250 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.894258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.894506 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.895172 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:44.895200 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:44.897424 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:44.900313 (+ 2889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:44.900319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:44.900330 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:44.900830 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:44.901259 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:44.901262 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:44.901264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:44.901446 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:44.902037 (+ 591us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:44.902042 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:44.902308 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 15:05:44.902511 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 15:05:44.902851 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":303,"threads_started":1}
I20260808 15:05:44.905395 24963 heartbeater.cc:347] Connected to a master server at 127.2.180.254:45669
I20260808 15:05:44.905697 24963 heartbeater.cc:464] Registering TS with master...
I20260808 15:05:44.905831 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.887012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45360 (local address 127.2.180.254:45669)
0808 15:05:44.892736 (+ 5724us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:44.892742 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:44.892760 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:05:44.892807 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:44.892812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:44.892884 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:44.892986 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:44.894482 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.895469 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.896854 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.897288 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.897379 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.900065 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:44.900072 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:44.900076 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:44.900129 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:44.903042 (+ 2913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:44.903796 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:44.903829 (+ 33us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:44.903832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:44.903952 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:44.904365 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:44.904370 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:44.904373 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:44.904718 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:44.904969 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:44.905273 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 15:05:44.905469 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5609,"thread_start_us":5478,"threads_started":1}
I20260808 15:05:44.906483 24963 heartbeater.cc:511] Master 127.2.180.254:45669 requested a full tablet report, sending...
I20260808 15:05:44.908023 24293 ts_manager.cc:194] Registered new tserver with Master: 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865)
I20260808 15:05:44.908860 24293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:40691
I20260808 15:05:44.920009 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:05:44.926105 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.920856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45374 (local address 127.2.180.254:45669)
0808 15:05:44.921004 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:44.921008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:44.921425 (+ 417us) server_negotiation.cc:407] Connection header received
0808 15:05:44.921615 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:44.921618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:44.921741 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:44.921850 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:05:44.922223 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.922893 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.924305 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.924690 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.924777 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.925233 (+ 456us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:05:44.925674 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:05:44.925772 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:44.925856 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 15:05:44.925915 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:05:44.926887 24293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:45374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:05:44.933837 24293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:05:44.953660 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.944557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:38865 (local address 127.0.0.1:33230)
0808 15:05:44.945401 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:44.945451 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:44.945569 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:44.946483 (+ 914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:44.946490 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:44.946521 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:44.946964 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.946978 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.948580 (+ 1602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.948585 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.950819 (+ 2234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.950834 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.952655 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.952662 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.952905 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.952959 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 15:05:44.953108 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 15:05:44.953300 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":469,"threads_started":1}
I20260808 15:05:44.954054 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.944809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.2.180.195:38865)
0808 15:05:44.945533 (+ 724us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:44.945541 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:44.945586 (+ 45us) server_negotiation.cc:407] Connection header received
0808 15:05:44.945756 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:44.945763 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:44.946000 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:44.946182 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:44.947188 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.948452 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.950984 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.952436 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.952559 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.952769 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:44.953638 (+ 869us) server_negotiation.cc:299] Negotiation successful
0808 15:05:44.953800 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":403,"threads_started":1}
I20260808 15:05:44.954203 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.945798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:39403 (local address 127.0.0.1:35770)
0808 15:05:44.946324 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:44.946342 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:44.946458 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:44.947507 (+ 1049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:44.947511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:44.947536 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:44.947868 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.947882 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.949688 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.949693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.951457 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.951469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.953425 (+ 1956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.953432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.953588 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.953620 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:05:44.953711 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 15:05:44.953835 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":274,"threads_started":1}
I20260808 15:05:44.955570 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.946131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.2.180.194:39403)
0808 15:05:44.946941 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:44.946948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:44.946982 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:05:44.947072 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:44.947079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:44.947307 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:44.948067 (+ 760us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:44.948341 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.949538 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.951934 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.953157 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.953314 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.953527 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:44.955041 (+ 1514us) server_negotiation.cc:299] Negotiation successful
0808 15:05:44.955220 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":509,"threads_started":1}
I20260808 15:05:44.955670 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.946396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38788)
0808 15:05:44.947251 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:44.947276 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:44.947378 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:44.948656 (+ 1278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:44.948660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:44.948686 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:44.949130 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.949143 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.950838 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.950843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.952898 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:44.952914 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.954910 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.954916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:44.955077 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.955108 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:05:44.955196 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 15:05:44.955307 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":4,"thread_start_us":678,"threads_started":1}
I20260808 15:05:44.956367 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:44.946881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38788 (local address 127.2.180.193:38445)
0808 15:05:44.948094 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:44.948102 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:44.948127 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:05:44.948218 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:44.948225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:44.948428 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:44.948673 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:44.949326 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.950688 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.953091 (+ 2403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:44.954691 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:44.954798 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:44.954978 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:44.955893 (+ 915us) server_negotiation.cc:299] Negotiation successful
0808 15:05:44.956061 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1056,"thread_start_us":834,"threads_started":1}
I20260808 15:05:44.958349 24874 tablet_service.cc:1467] Processing CreateTablet for tablet d31e3c75698f47959ce83d959fef315f (DEFAULT_TABLE table=test-table [id=cee3890f010b497abda73f920c59b6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:44.959071 24664 tablet_service.cc:1467] Processing CreateTablet for tablet d31e3c75698f47959ce83d959fef315f (DEFAULT_TABLE table=test-table [id=cee3890f010b497abda73f920c59b6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:44.959394 24475 tablet_service.cc:1467] Processing CreateTablet for tablet d31e3c75698f47959ce83d959fef315f (DEFAULT_TABLE table=test-table [id=cee3890f010b497abda73f920c59b6e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:05:44.959890 24874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d31e3c75698f47959ce83d959fef315f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:44.960467 24664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d31e3c75698f47959ce83d959fef315f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:44.960618 24475 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d31e3c75698f47959ce83d959fef315f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:05:44.972189 24982 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Bootstrap starting.
I20260808 15:05:44.975121 24984 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: Bootstrap starting.
I20260808 15:05:44.975719 24982 tablet_bootstrap.cc:654] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:44.976035 24983 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: Bootstrap starting.
I20260808 15:05:44.977080 24982 log.cc:824] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:44.979095 24984 tablet_bootstrap.cc:654] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:44.979247 24983 tablet_bootstrap.cc:654] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: Neither blocks nor log segments found. Creating new log.
I20260808 15:05:44.980180 24984 log.cc:824] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:44.980456 24983 log.cc:824] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:44.981316 24982 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: No bootstrap required, opened a new log
I20260808 15:05:44.981552 24982 ts_tablet_manager.cc:1397] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 15:05:44.982630 24984 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: No bootstrap required, opened a new log
I20260808 15:05:44.982847 24984 ts_tablet_manager.cc:1397] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260808 15:05:44.984532 24983 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: No bootstrap required, opened a new log
I20260808 15:05:44.984799 24983 ts_tablet_manager.cc:1397] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 15:05:44.989673 24984 raft_consensus.cc:348] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:44.989990 24984 raft_consensus.cc:374] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:44.990087 24984 raft_consensus.cc:729] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12fd35dc0b144c29beced2c53e83d920, State: Initialized, Role: FOLLOWER
I20260808 15:05:44.990614 24984 consensus_queue.cc:260] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:44.990975 24982 raft_consensus.cc:348] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:44.991379 24982 raft_consensus.cc:374] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:44.991482 24982 raft_consensus.cc:729] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1659d7ee5a4d45a51c0ae1cec677c6, State: Initialized, Role: FOLLOWER
I20260808 15:05:44.991832 24963 heartbeater.cc:503] Master 127.2.180.254:45669 was elected leader, sending a full tablet report...
I20260808 15:05:44.992144 24982 consensus_queue.cc:260] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:44.992364 24984 ts_tablet_manager.cc:1428] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: Time spent starting tablet: real 0.009s user 0.008s sys 0.001s
I20260808 15:05:44.993336 24568 heartbeater.cc:503] Master 127.2.180.254:45669 was elected leader, sending a full tablet report...
I20260808 15:05:44.993875 24982 ts_tablet_manager.cc:1428] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260808 15:05:44.995342 24983 raft_consensus.cc:348] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:44.995743 24983 raft_consensus.cc:374] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:05:44.995857 24983 raft_consensus.cc:729] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcf60a19737b46148e6d83fb2d818683, State: Initialized, Role: FOLLOWER
I20260808 15:05:44.996395 24983 consensus_queue.cc:260] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:44.997427 24769 heartbeater.cc:503] Master 127.2.180.254:45669 was elected leader, sending a full tablet report...
I20260808 15:05:44.998095 24983 ts_tablet_manager.cc:1428] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260808 15:05:45.002874 24569 tablet.cc:2369] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:05:45.015542 24992 raft_consensus.cc:482] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:05:45.015758 24992 raft_consensus.cc:504] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:45.017134 24992 leader_election.cc:288] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865), 0d1659d7ee5a4d45a51c0ae1cec677c6 (127.2.180.193:38445)
I20260808 15:05:45.023921 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.017410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:49381 (local address 127.2.180.195:38865)
0808 15:05:45.017566 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:45.017570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:45.018079 (+ 509us) server_negotiation.cc:407] Connection header received
0808 15:05:45.018259 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:45.018264 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:45.018338 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:45.018418 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:45.019058 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.020022 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.022043 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.022970 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.023085 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:45.023145 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:45.023539 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 15:05:45.023664 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:05:45.023965 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.017426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:38865 (local address 127.2.180.194:49381)
0808 15:05:45.017986 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.018001 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:45.018124 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:45.018475 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:45.018480 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:45.018509 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:45.018930 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:45.018944 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.020141 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.020146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:45.021877 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:45.021896 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.023191 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.023198 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:45.023359 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:45.023393 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 15:05:45.023488 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 15:05:45.023601 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"mutex_wait_us":1,"thread_start_us":298,"threads_started":1}
I20260808 15:05:45.024220 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.018184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:40941 (local address 127.2.180.193:38445)
0808 15:05:45.018367 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:45.018373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:45.018756 (+ 383us) server_negotiation.cc:407] Connection header received
0808 15:05:45.018914 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:45.018918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:45.018986 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:45.019078 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:05:45.019617 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.020506 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.022431 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.023254 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.023401 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:45.023467 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:45.023863 (+ 396us) server_negotiation.cc:299] Negotiation successful
0808 15:05:45.023989 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:05:45.024436 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.018076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.2.180.194:40941)
0808 15:05:45.018654 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.018671 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:45.018799 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:45.019095 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:45.019100 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:45.019126 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:05:45.019487 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:45.019499 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.020654 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.020658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:45.022275 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:45.022290 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.023454 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.023460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:45.023664 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:45.023702 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 15:05:45.023795 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 15:05:45.023956 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"mutex_wait_us":23,"thread_start_us":249,"threads_started":1}
I20260808 15:05:45.025071 24901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d31e3c75698f47959ce83d959fef315f" candidate_uuid: "dcf60a19737b46148e6d83fb2d818683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12fd35dc0b144c29beced2c53e83d920" is_pre_election: true
I20260808 15:05:45.025347 24502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d31e3c75698f47959ce83d959fef315f" candidate_uuid: "dcf60a19737b46148e6d83fb2d818683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" is_pre_election: true
I20260808 15:05:45.025581 24901 raft_consensus.cc:2454] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcf60a19737b46148e6d83fb2d818683 in term 0.
I20260808 15:05:45.025851 24502 raft_consensus.cc:2454] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcf60a19737b46148e6d83fb2d818683 in term 0.
I20260808 15:05:45.026264 24594 leader_election.cc:302] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12fd35dc0b144c29beced2c53e83d920, dcf60a19737b46148e6d83fb2d818683; no voters:
I20260808 15:05:45.026614 24992 raft_consensus.cc:2781] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:05:45.026736 24992 raft_consensus.cc:482] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:05:45.026839 24992 raft_consensus.cc:3037] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:45.028942 24992 raft_consensus.cc:504] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:45.029480 24992 leader_election.cc:288] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [CANDIDATE]: Term 1 election: Requested vote from peers 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865), 0d1659d7ee5a4d45a51c0ae1cec677c6 (127.2.180.193:38445)
I20260808 15:05:45.029843 24901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d31e3c75698f47959ce83d959fef315f" candidate_uuid: "dcf60a19737b46148e6d83fb2d818683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12fd35dc0b144c29beced2c53e83d920"
I20260808 15:05:45.029924 24502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d31e3c75698f47959ce83d959fef315f" candidate_uuid: "dcf60a19737b46148e6d83fb2d818683" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6"
I20260808 15:05:45.030004 24901 raft_consensus.cc:3037] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:45.030098 24502 raft_consensus.cc:3037] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:05:45.032182 24901 raft_consensus.cc:2454] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcf60a19737b46148e6d83fb2d818683 in term 1.
I20260808 15:05:45.032251 24502 raft_consensus.cc:2454] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcf60a19737b46148e6d83fb2d818683 in term 1.
I20260808 15:05:45.032557 24594 leader_election.cc:302] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12fd35dc0b144c29beced2c53e83d920, dcf60a19737b46148e6d83fb2d818683; no voters:
I20260808 15:05:45.032830 24992 raft_consensus.cc:2781] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:05:45.033226 24992 raft_consensus.cc:686] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 1 LEADER]: Becoming Leader. State: Replica: dcf60a19737b46148e6d83fb2d818683, State: Running, Role: LEADER
I20260808 15:05:45.033627 24992 consensus_queue.cc:237] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:45.037710 24293 catalog_manager.cc:5495] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 reported cstate change: term changed from 0 to 1, leader changed from <none> to dcf60a19737b46148e6d83fb2d818683 (127.2.180.194). New cstate: current_term: 1 leader_uuid: "dcf60a19737b46148e6d83fb2d818683" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } health_report { overall_health: UNKNOWN } } }
I20260808 15:05:45.055672 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.050971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35776 (local address 127.2.180.194:39403)
0808 15:05:45.051130 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:45.051134 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:45.051150 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:05:45.051297 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:45.051300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:45.051365 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:45.051477 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:05:45.051783 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.052524 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.053892 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.054284 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.054373 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:45.054792 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:05:45.055211 (+ 419us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:05:45.055309 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:45.055384 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 15:05:45.055442 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:05:45.062783 24901 raft_consensus.cc:1261] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 1 FOLLOWER]: Refusing update from remote peer dcf60a19737b46148e6d83fb2d818683: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:05:45.062830 24502 raft_consensus.cc:1261] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Refusing update from remote peer dcf60a19737b46148e6d83fb2d818683: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:05:45.063555 24995 consensus_queue.cc:1035] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [LEADER]: Connected to new peer: Peer: permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:05:45.063932 24992 consensus_queue.cc:1035] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:05:45.087342 25002 mvcc.cc:205] Tried to move back new op lower bound from 7316281528568750080 to 7316281528461090816. Current Snapshot: MvccSnapshot[applied={T|T < 7316281528568750080}]
I20260808 15:05:45.096349 24293 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:45374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:05:45.096617 24293 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:45374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:05:45.100411 24293 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Sending DeleteTablet for 3 replicas of tablet d31e3c75698f47959ce83d959fef315f
I20260808 15:05:45.101531 24475 tablet_service.cc:1514] Processing DeleteTablet for tablet d31e3c75698f47959ce83d959fef315f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:45 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:38788
I20260808 15:05:45.101713 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 24356
I20260808 15:05:45.101863 24664 tablet_service.cc:1514] Processing DeleteTablet for tablet d31e3c75698f47959ce83d959fef315f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:45 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:35770
I20260808 15:05:45.102375 24874 tablet_service.cc:1514] Processing DeleteTablet for tablet d31e3c75698f47959ce83d959fef315f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:05:45 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33230
I20260808 15:05:45.113304 25015 tablet_replica.cc:330] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: stopping tablet replica
I20260808 15:05:45.113729 25015 raft_consensus.cc:2229] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:05:45.114250 25015 raft_consensus.cc:2258] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:45.116271 25017 tablet_replica.cc:330] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: stopping tablet replica
I20260808 15:05:45.116597 25017 raft_consensus.cc:2229] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:05:45.118268 25017 raft_consensus.cc:2258] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:45.119315 25015 ts_tablet_manager.cc:1905] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:05:45.119612 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.102775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.2.180.193:38445)
0808 15:05:45.102947 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:45.102954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:45.102974 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:05:45.103133 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:45.103138 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:45.103208 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:45.103311 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:45.105127 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.106365 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.108416 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.109129 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.109254 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:45.112705 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:45.112719 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:45.112734 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:45.112779 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:45.116491 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:45.117302 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:45.117311 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:45.117315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:45.117467 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:45.117872 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:45.117878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:45.117881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:45.118340 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:45.118637 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:45.119037 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 15:05:45.119259 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:05:45.120433 24562 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:45.122840 25017 ts_tablet_manager.cc:1905] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:05:45.129321 25015 ts_tablet_manager.cc:1918] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:05:45.136651 25017 ts_tablet_manager.cc:1918] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:05:45.136920 25015 log.cc:1196] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/d31e3c75698f47959ce83d959fef315f
I20260808 15:05:45.137571 25015 ts_tablet_manager.cc:1939] T d31e3c75698f47959ce83d959fef315f P dcf60a19737b46148e6d83fb2d818683: Deleting consensus metadata
I20260808 15:05:45.139348 25017 log.cc:1196] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/d31e3c75698f47959ce83d959fef315f
I20260808 15:05:45.139544 24269 catalog_manager.cc:4841] TS dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403): tablet d31e3c75698f47959ce83d959fef315f (table test-table [id=cee3890f010b497abda73f920c59b6e2]) successfully deleted
I20260808 15:05:45.140023 25017 ts_tablet_manager.cc:1939] T d31e3c75698f47959ce83d959fef315f P 12fd35dc0b144c29beced2c53e83d920: Deleting consensus metadata
I20260808 15:05:45.141755 24269 catalog_manager.cc:4841] TS 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865): tablet d31e3c75698f47959ce83d959fef315f (table test-table [id=cee3890f010b497abda73f920c59b6e2]) successfully deleted
I20260808 15:05:45.226378 25014 tablet_replica.cc:330] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: stopping tablet replica
I20260808 15:05:45.226819 25014 raft_consensus.cc:2229] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:05:45.227308 25014 raft_consensus.cc:2258] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:05:45.228832 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 24356
I20260808 15:05:45.252496 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 24575
W20260808 15:05:45.252507 24269 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.180.193:38445 (error 108)
I20260808 15:05:45.254757 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.254373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38808)
0808 15:05:45.254523 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.254620 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:38445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 15:05:45.255066 24269 catalog_manager.cc:4584] TS 0d1659d7ee5a4d45a51c0ae1cec677c6 (127.2.180.193:38445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d31e3c75698f47959ce83d959fef315f: Network error: Client connection negotiation failed: client connection to 127.2.180.193:38445: connect: Connection refused (error 111)
I20260808 15:05:45.267652 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.253327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35786 (local address 127.2.180.194:39403)
0808 15:05:45.253614 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:45.253621 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:45.253676 (+ 55us) server_negotiation.cc:407] Connection header received
0808 15:05:45.254627 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:45.254633 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:45.254719 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:45.254822 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:45.256266 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.257458 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.258809 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.259270 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.259388 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:45.262005 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:45.262017 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:45.262030 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:45.262067 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:45.264961 (+ 2894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:45.265621 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:45.265628 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:45.265630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:45.265776 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:45.266141 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:45.266145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:45.266147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:45.266494 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:45.266811 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:45.267124 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 15:05:45.267339 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260808 15:05:45.268339 24764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:45.308718 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.308234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38816)
0808 15:05:45.308429 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.308537 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:38445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 15:05:45.309728 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.309334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38824)
0808 15:05:45.309483 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.309579 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:38445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 15:05:45.370293 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.369674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38840)
0808 15:05:45.369944 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.370061 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:38445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260808 15:05:45.371294 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.370943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38848)
0808 15:05:45.371076 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.371167 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:38445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 15:05:45.379969 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 24575
I20260808 15:05:45.403836 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 24772
I20260808 15:05:45.416867 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.404562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.2.180.195:38865)
0808 15:05:45.404755 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:45.404760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:45.404778 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:05:45.404923 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:45.404929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:45.404997 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:45.405080 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:45.406400 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.407124 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.408503 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:45.408911 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:45.409023 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:45.411552 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:45.411567 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:45.411584 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:45.411625 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:45.414169 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:45.414882 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:45.414889 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:45.414891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:45.415048 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:45.415396 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:45.415401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:45.415404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:45.415740 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:45.416053 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:45.416347 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 15:05:45.416558 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 15:05:45.417472 24957 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:45.451993 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.451417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38854)
0808 15:05:45.451653 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.451758 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:38445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 15:05:45.453083 24979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:45.452634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38445 (local address 127.0.0.1:38860)
0808 15:05:45.452847 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:45.452934 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:38445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260808 15:05:45.505833 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 24772
I20260808 15:05:45.530576 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 24251
I20260808 15:05:45.531711 24347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:05:45.641067 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 24251
I20260808 15:05:45.664682 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45669
--webserver_interface=127.2.180.254
--webserver_port=36285
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:45669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:45.982342 25027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:45.982604 25027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:45.982698 25027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:45.989596 25027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:05:45.989694 25027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:45.989745 25027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:45.989800 25027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:05:45.989843 25027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:05:45.998986 25027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:45669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:45669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=36285
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:46.000135 25027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:46.001286 25027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:46.012039 25036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:46.012077 25037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:46.011991 25039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:46.012902 25027 server_base.cc:1034] running on GCE node
I20260808 15:05:46.013607 25027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:46.014843 25027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:46.016038 25027 hybrid_clock.cc:648] HybridClock initialized: now 1786201546015982 us; error 40 us; skew 500 ppm
I20260808 15:05:46.016335 25027 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:46 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:46.019188 25027 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:05:46.020605 25027 webserver.cc:460] Webserver started at http://127.2.180.254:36285/ using document root <none> and password file <none>
I20260808 15:05:46.021194 25027 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:46.021293 25027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:46.026270 25027 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:05:46.028896 25048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:46.030078 25027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:05:46.030234 25027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "4b04a0a6e99cfd6146341d80ab6ec5e4"
server_key_iv: "b5a624d91a81ff3541deabcf003756ec"
server_key_version: "encryptionkey@0"
I20260808 15:05:46.031131 25027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:46.056200 25027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:46.059744 25027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:46.059988 25027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:46.076905 25027 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:45669
I20260808 15:05:46.076936 25123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:45669 every 8 connection(s)
I20260808 15:05:46.078193 25027 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:05:46.084532 25124 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:05:46.084935 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 25027
I20260808 15:05:46.085928 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:38445
--local_ip_for_outbound_sockets=127.2.180.193
--tserver_master_addrs=127.2.180.254:45669
--webserver_port=45697
--webserver_interface=127.2.180.193
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:05:46.088750 25124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Bootstrap starting.
I20260808 15:05:46.109539 25124 log.cc:824] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:46.124907 25124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:05:46.125381 25124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3: Bootstrap complete.
I20260808 15:05:46.133981 25124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:46.134853 25124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0700fc6a2ba3417a8e4f9a1cdb902de3, State: Initialized, Role: FOLLOWER
I20260808 15:05:46.135434 25124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:46.135658 25124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:05:46.135766 25124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:05:46.135951 25124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 2 FOLLOWER]: Advancing to term 3
I20260808 15:05:46.139521 25124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:46.140030 25124 leader_election.cc:302] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0700fc6a2ba3417a8e4f9a1cdb902de3; no voters:
I20260808 15:05:46.140633 25124 leader_election.cc:288] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 15:05:46.140780 25128 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 15:05:46.141750 25128 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [term 3 LEADER]: Becoming Leader. State: Replica: 0700fc6a2ba3417a8e4f9a1cdb902de3, State: Running, Role: LEADER
I20260808 15:05:46.142282 25128 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } }
I20260808 15:05:46.142690 25124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:05:46.144623 25130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0700fc6a2ba3417a8e4f9a1cdb902de3. Latest consensus state: current_term: 3 leader_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } } }
I20260808 15:05:46.145010 25130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:46.145507 25129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0700fc6a2ba3417a8e4f9a1cdb902de3" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 45669 } } }
I20260808 15:05:46.145778 25129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0700fc6a2ba3417a8e4f9a1cdb902de3 [sys.catalog]: This master's current role is: LEADER
I20260808 15:05:46.148994 25135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:05:46.160442 25135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:05:46.161896 25135 catalog_manager.cc:1259] Loaded cluster ID: 0a3fea95ee0a4a5ea0399853378587d0
I20260808 15:05:46.162051 25135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 15:05:46.164175 25135 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fb76f02b9ad google::LogMessage::Fail() at ??:0
@ 0x7fb76f02e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fb76f02b4de google::LogMessage::Flush() at ??:0
@ 0x7fb76f02ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb784a5e512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fb784a5b976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fb784ae3610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fb784ae3439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb771c77345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fb771c7a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fb771c7a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fb771c55eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fb774f2f6db start_thread at ??:0
@ 0x7fb76dec561f clone at ??:0
W20260808 15:05:46.454510 25126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:46.454799 25126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:46.454936 25126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:46.462009 25126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:46.462126 25126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:46.462278 25126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:05:46.471410 25126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:38445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=45697
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:46.472605 25126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:46.473853 25126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:46.486300 25180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:46.486249 25185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:46.486336 25183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:46.487326 25126 server_base.cc:1034] running on GCE node
I20260808 15:05:46.487959 25126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:46.488802 25126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:46.490001 25126 hybrid_clock.cc:648] HybridClock initialized: now 1786201546489955 us; error 34 us; skew 500 ppm
I20260808 15:05:46.490288 25126 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:46 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:46.493669 25126 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:05:46.495415 25126 webserver.cc:460] Webserver started at http://127.2.180.193:45697/ using document root <none> and password file <none>
I20260808 15:05:46.496290 25126 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:46.496425 25126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:46.501616 25126 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:05:46.504390 25197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:46.505755 25126 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:05:46.505920 25126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6"
format_stamp: "Formatted at 2026-08-08 15:05:40 on dist-test-slave-jlzn"
server_key: "d3542d670e831a225b21f4e36e27e7e5"
server_key_iv: "f9e841c2d26e28b2283ca53ac18a6382"
server_key_version: "encryptionkey@0"
I20260808 15:05:46.506853 25126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:46.525269 25126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:46.528813 25126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:46.529049 25126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:46.529944 25126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:46.531605 25206 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 15:05:46.536958 25126 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 15:05:46.537079 25126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 15:05:46.537200 25126 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 15:05:46.540202 25126 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 15:05:46.540279 25126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260808 15:05:46.540587 25206 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Bootstrap starting.
I20260808 15:05:46.548309 25206 log.cc:824] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Log is configured to *not* fsync() on all Append() calls
I20260808 15:05:46.562333 25206 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:05:46.563227 25206 tablet_bootstrap.cc:492] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Bootstrap complete.
I20260808 15:05:46.563733 25206 ts_tablet_manager.cc:1397] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Time spent bootstrapping tablet: real 0.023s user 0.014s sys 0.008s
I20260808 15:05:46.572878 25206 raft_consensus.cc:348] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:46.573809 25206 raft_consensus.cc:729] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d1659d7ee5a4d45a51c0ae1cec677c6, State: Initialized, Role: FOLLOWER
I20260808 15:05:46.574435 25206 consensus_queue.cc:260] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:46.575932 25206 ts_tablet_manager.cc:1428] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6: Time spent starting tablet: real 0.012s user 0.009s sys 0.003s
I20260808 15:05:46.582183 25126 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:38445
I20260808 15:05:46.582237 25368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:38445 every 8 connection(s)
I20260808 15:05:46.583319 25126 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:05:46.587715 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:46.585779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:52423)
0808 15:05:46.586667 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:46.586926 (+ 259us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":382,"threads_started":1}
W20260808 15:05:46.588312 25370 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:45669 (0 consecutive failures): Network error: Failed to ping master at 127.2.180.254:45669: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
I20260808 15:05:46.589020 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:46.588740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:38391)
0808 15:05:46.588855 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:46.588920 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260808 15:05:46.589736 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:46.589437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:35007)
0808 15:05:46.589548 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:46.589627 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 15:05:46.589936 25370 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 15:05:46.593077 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 25126
I20260808 15:05:46.593659 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:39403
--local_ip_for_outbound_sockets=127.2.180.194
--tserver_master_addrs=127.2.180.254:45669
--webserver_port=35719
--webserver_interface=127.2.180.194
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:05:46.912775 25376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:46.913034 25376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:46.913151 25376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:46.920071 25376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:46.920173 25376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:46.920315 25376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:05:46.929353 25376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:39403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=35719
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:46.930420 25376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:46.931617 25376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:46.943608 25390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:46.943682 25387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:46.943675 25385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:46.944520 25376 server_base.cc:1034] running on GCE node
I20260808 15:05:46.945077 25376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:46.945910 25376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:46.947108 25376 hybrid_clock.cc:648] HybridClock initialized: now 1786201546947045 us; error 54 us; skew 500 ppm
I20260808 15:05:46.947377 25376 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:46 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:46.950314 25376 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:05:46.951663 25376 webserver.cc:460] Webserver started at http://127.2.180.194:35719/ using document root <none> and password file <none>
I20260808 15:05:46.952311 25376 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:46.952425 25376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:46.957304 25376 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:05:46.959743 25406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:46.960979 25376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:05:46.961125 25376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "dcf60a19737b46148e6d83fb2d818683"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "c6ed5e386421d99824e5348dc0dc31bd"
server_key_iv: "40a8f01e729b42913155b4ceaf7abe76"
server_key_version: "encryptionkey@0"
I20260808 15:05:46.962071 25376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:46.989447 25376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:46.992956 25376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:46.993175 25376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:46.993999 25376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:46.995185 25376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:46.995270 25376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:46.995344 25376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:46.995398 25376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:47.023113 25376 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:39403
I20260808 15:05:47.023170 25568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:39403 every 8 connection(s)
I20260808 15:05:47.024416 25376 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:05:47.027395 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 25376
I20260808 15:05:47.028154 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:38865
--local_ip_for_outbound_sockets=127.2.180.195
--tserver_master_addrs=127.2.180.254:45669
--webserver_port=41403
--webserver_interface=127.2.180.195
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:40933
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:05:47.028931 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.027027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:49403)
0808 15:05:47.027857 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.028109 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":325,"threads_started":1}
W20260808 15:05:47.029600 25569 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:45669 (0 consecutive failures): Network error: Failed to ping master at 127.2.180.254:45669: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
I20260808 15:05:47.030320 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.030017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:53517)
0808 15:05:47.030152 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.030220 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260808 15:05:47.031101 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.030806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:56879)
0808 15:05:47.030914 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.030984 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 15:05:47.031327 25569 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:05:47.339634 25573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:05:47.339905 25573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:05:47.340006 25573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:05:47.346791 25573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:05:47.346884 25573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:05:47.346993 25573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:05:47.356062 25573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:40933
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:38865
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=41403
--tserver_master_addrs=127.2.180.254:45669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:05:47.357153 25573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:05:47.358340 25573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:05:47.370321 25585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:47.370282 25593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:05:47.370359 25586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:05:47.371323 25573 server_base.cc:1034] running on GCE node
I20260808 15:05:47.371923 25573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:05:47.372748 25573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:05:47.373939 25573 hybrid_clock.cc:648] HybridClock initialized: now 1786201547373889 us; error 45 us; skew 500 ppm
I20260808 15:05:47.374219 25573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:05:47 dist-test-slave-jlzn krb5kdc[23445](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201547, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:05:47.377161 25573 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:05:47.378433 25573 webserver.cc:460] Webserver started at http://127.2.180.195:41403/ using document root <none> and password file <none>
I20260808 15:05:47.379052 25573 fs_manager.cc:359] Metadata directory not provided
I20260808 15:05:47.379166 25573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:05:47.384040 25573 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:05:47.386507 25608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:47.387681 25573 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:05:47.387872 25573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "12fd35dc0b144c29beced2c53e83d920"
format_stamp: "Formatted at 2026-08-08 15:05:41 on dist-test-slave-jlzn"
server_key: "d48da4044197590463700fbf3248ecf9"
server_key_iv: "ed47f3daf376d0b72c88376f22a3bc02"
server_key_version: "encryptionkey@0"
I20260808 15:05:47.388777 25573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:05:47.405676 25573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:05:47.409453 25573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:05:47.409689 25573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:05:47.410586 25573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:05:47.411851 25573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:05:47.411947 25573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:47.412060 25573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:05:47.412130 25573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:05:47.441767 25573 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:38865
I20260808 15:05:47.441828 25761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:38865 every 8 connection(s)
I20260808 15:05:47.443117 25573 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:05:47.447566 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.445824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:47971)
0808 15:05:47.446511 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.446737 (+ 226us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":262,"threads_started":1}
W20260808 15:05:47.448253 25762 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:45669 (0 consecutive failures): Network error: Failed to ping master at 127.2.180.254:45669: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
I20260808 15:05:47.448974 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.448668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:36965)
0808 15:05:47.448806 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.448870 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 15:05:47.449741 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.449437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:60951)
0808 15:05:47.449572 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.449643 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 15:05:47.449963 25762 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 15:05:47.450934 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 25573
I20260808 15:05:47.591053 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.590371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:55057)
0808 15:05:47.590765 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.590846 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":192,"threads_started":1}
I20260808 15:05:47.994074 25767 raft_consensus.cc:482] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:05:47.994342 25767 raft_consensus.cc:504] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:47.995961 25767 leader_election.cc:288] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865), dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
Aug 08 15:05:48 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
Aug 08 15:05:48 dist-test-slave-jlzn krb5kdc[23445](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201546, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:05:48.020192 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.996266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:38865 (local address 127.2.180.193:39249)
0808 15:05:47.996457 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.996503 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:47.997096 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:47.999700 (+ 2604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:47.999711 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:48.000752 (+ 1041us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:48.001832 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:48.001859 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:48.004075 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:48.004081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:48.006262 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:48.006278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:48.007578 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:48.007584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:48.007974 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:48.008773 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:48.008803 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:48.012272 (+ 3469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:48.016032 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:48.016041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:48.016056 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:48.016802 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:48.017289 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:48.017294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:48.017298 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:48.017550 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:48.018432 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:48.018440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:48.018872 (+ 432us) client_negotiation.cc:769] Sending connection context
0808 15:05:48.019205 (+ 333us) client_negotiation.cc:240] Negotiation successful
0808 15:05:48.019841 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":501,"spinlock_wait_cycles":48384}
I20260808 15:05:48.020294 25774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.996966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:53339 (local address 127.2.180.194:39403)
0808 15:05:47.997584 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:47.997593 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:47.997623 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:05:47.998984 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:47.998992 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:47.999323 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:47.999629 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:48.002046 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:48.003943 (+ 1897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:48.006409 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:48.007396 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:48.007593 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:48.012244 (+ 4651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:48.012252 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:48.012264 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:48.012298 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:48.015896 (+ 3598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:48.016941 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:48.016950 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:48.016958 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:48.017179 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:48.017662 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:48.017666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:48.017670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:48.018309 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:48.018490 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:48.019448 (+ 958us) server_negotiation.cc:299] Negotiation successful
0808 15:05:48.019869 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":364,"threads_started":1}
I20260808 15:05:48.020294 25771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.996368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:39249 (local address 127.2.180.195:38865)
0808 15:05:47.997145 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0808 15:05:47.997162 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 15:05:47.997209 (+ 47us) server_negotiation.cc:407] Connection header received
0808 15:05:47.998910 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:05:47.998919 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:05:47.999269 (+ 350us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:05:47.999509 (+ 240us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:05:48.002054 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:48.003896 (+ 1842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:48.006409 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:05:48.007396 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:05:48.007584 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:48.012522 (+ 4938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:05:48.012531 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:05:48.012546 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:05:48.012593 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:05:48.015896 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:48.016941 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:48.016950 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:48.016958 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:48.017179 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:05:48.017662 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:05:48.017666 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:05:48.017670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:05:48.018303 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:05:48.018486 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 15:05:48.019448 (+ 962us) server_negotiation.cc:299] Negotiation successful
0808 15:05:48.019869 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":486,"threads_started":1}
I20260808 15:05:48.020192 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:47.996604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:39403 (local address 127.2.180.193:53339)
0808 15:05:47.997455 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:47.997471 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:05:47.997570 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:05:47.999701 (+ 2131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:05:47.999711 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:05:48.000752 (+ 1041us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:05:48.001832 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:48.001859 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:48.004075 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:48.004081 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:48.006262 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:05:48.006278 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:05:48.007613 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:05:48.007621 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:05:48.007968 (+ 347us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:05:48.008788 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:05:48.008815 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:05:48.011964 (+ 3149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:05:48.016032 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:48.016041 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:48.016056 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:48.016802 (+ 746us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:48.017289 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:05:48.017294 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:05:48.017298 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:05:48.017550 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:05:48.018432 (+ 882us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:05:48.018440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:05:48.018872 (+ 432us) client_negotiation.cc:769] Sending connection context
0808 15:05:48.019205 (+ 333us) client_negotiation.cc:240] Negotiation successful
0808 15:05:48.019685 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"mutex_wait_us":412,"spinlock_wait_cycles":66048,"thread_start_us":563,"threads_started":1}
W20260808 15:05:48.023967 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
W20260808 15:05:48.024308 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
I20260808 15:05:48.024425 25198 leader_election.cc:302] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d1659d7ee5a4d45a51c0ae1cec677c6; no voters: 12fd35dc0b144c29beced2c53e83d920, dcf60a19737b46148e6d83fb2d818683
I20260808 15:05:48.024788 25767 raft_consensus.cc:2726] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:05:48.032492 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:48.031715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:54785)
0808 15:05:48.032162 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:48.032289 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":228,"threads_started":1}
I20260808 15:05:48.451333 25779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:48.450455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:56545)
0808 15:05:48.450971 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:48.451100 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":275,"threads_started":1}
I20260808 15:05:48.592468 25780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:48.591787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:38709)
0808 15:05:48.592211 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:48.592291 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":188,"threads_started":1}
I20260808 15:05:49.033874 25782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:49.033244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:40623)
0808 15:05:49.033645 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:49.033730 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":212,"threads_started":1}
I20260808 15:05:49.452731 25783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:49.452093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:59365)
0808 15:05:49.452490 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:49.452567 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":204,"threads_started":1}
I20260808 15:05:49.593925 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:49.593223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:44573)
0808 15:05:49.593666 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:49.593744 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":211,"threads_started":1}
I20260808 15:05:50.035257 25785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:50.034617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:51691)
0808 15:05:50.035016 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:50.035104 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":197,"threads_started":1}
I20260808 15:05:50.131232 25786 raft_consensus.cc:482] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:05:50.131404 25786 raft_consensus.cc:504] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:50.132004 25786 leader_election.cc:288] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865), dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
W20260808 15:05:50.132958 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
W20260808 15:05:50.133186 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
I20260808 15:05:50.133272 25198 leader_election.cc:302] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d1659d7ee5a4d45a51c0ae1cec677c6; no voters: 12fd35dc0b144c29beced2c53e83d920, dcf60a19737b46148e6d83fb2d818683
I20260808 15:05:50.133525 25786 raft_consensus.cc:2726] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:05:50.454085 25789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:50.453448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:50099)
0808 15:05:50.453857 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:50.453935 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":211,"threads_started":1}
I20260808 15:05:50.595396 25790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:50.594672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:59697)
0808 15:05:50.595121 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:50.595217 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":203,"threads_started":1}
I20260808 15:05:51.036701 25791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:51.036025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:55329)
0808 15:05:51.036440 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:51.036517 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":209,"threads_started":1}
I20260808 15:05:51.455514 25792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:51.454833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:41823)
0808 15:05:51.455267 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:51.455344 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":223,"threads_started":1}
I20260808 15:05:51.597132 25794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:51.596254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:36833)
0808 15:05:51.596805 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:51.596912 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":250,"threads_started":1}
I20260808 15:05:52.038198 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:52.037547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:60867)
0808 15:05:52.037952 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:52.038033 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":206,"threads_started":1}
I20260808 15:05:52.457070 25796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:52.456366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:36099)
0808 15:05:52.456819 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:52.456900 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":247,"threads_started":1}
I20260808 15:05:52.598654 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:52.597905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:37163)
0808 15:05:52.598338 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:52.598454 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":232,"threads_started":1}
I20260808 15:05:52.888527 25800 raft_consensus.cc:482] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:05:52.888738 25800 raft_consensus.cc:504] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:52.889441 25800 leader_election.cc:288] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865), dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
W20260808 15:05:52.890393 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
W20260808 15:05:52.890594 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
I20260808 15:05:52.890704 25198 leader_election.cc:302] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d1659d7ee5a4d45a51c0ae1cec677c6; no voters: 12fd35dc0b144c29beced2c53e83d920, dcf60a19737b46148e6d83fb2d818683
I20260808 15:05:52.890954 25800 raft_consensus.cc:2726] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:05:53.039672 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:53.038921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:50747)
0808 15:05:53.039403 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:53.039502 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":273,"threads_started":1}
I20260808 15:05:53.458475 25802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:53.457842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:58301)
0808 15:05:53.458240 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:53.458317 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":206,"threads_started":1}
I20260808 15:05:53.600210 25803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:53.599443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:40371)
0808 15:05:53.599925 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:53.600013 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":256,"threads_started":1}
I20260808 15:05:54.041317 25804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:54.040510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:55639)
0808 15:05:54.041038 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:54.041129 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":218,"threads_started":1}
I20260808 15:05:54.460084 25805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:54.459303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:40719)
0808 15:05:54.459777 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:54.459897 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":209,"threads_started":1}
I20260808 15:05:54.601583 25807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:54.600953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:47371)
0808 15:05:54.601343 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:54.601423 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":206,"threads_started":1}
I20260808 15:05:55.042791 25808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:55.042106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:41689)
0808 15:05:55.042530 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:55.042612 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":213,"threads_started":1}
I20260808 15:05:55.461494 25810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:55.460824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:60735)
0808 15:05:55.461234 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:55.461316 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":208,"threads_started":1}
I20260808 15:05:55.603060 25811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:55.602345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:41925)
0808 15:05:55.602807 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:55.602888 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":236,"threads_started":1}
I20260808 15:05:56.044382 25812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:56.043537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:47647)
0808 15:05:56.044105 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:56.044200 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":287,"threads_started":1}
I20260808 15:05:56.462973 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:56.462307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:35527)
0808 15:05:56.462745 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:56.462825 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":205,"threads_started":1}
I20260808 15:05:56.604635 25814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:56.603937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:46873)
0808 15:05:56.604385 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:56.604477 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":228,"threads_started":1}
I20260808 15:05:56.649616 25818 raft_consensus.cc:482] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:05:56.649860 25818 raft_consensus.cc:504] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:05:56.650491 25818 leader_election.cc:288] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865), dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
W20260808 15:05:56.651360 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
W20260808 15:05:56.651571 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
I20260808 15:05:56.651651 25198 leader_election.cc:302] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d1659d7ee5a4d45a51c0ae1cec677c6; no voters: 12fd35dc0b144c29beced2c53e83d920, dcf60a19737b46148e6d83fb2d818683
I20260808 15:05:56.651899 25818 raft_consensus.cc:2726] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:05:57.045840 25819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:57.045168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:60389)
0808 15:05:57.045584 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:57.045671 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":227,"threads_started":1}
I20260808 15:05:57.464475 25821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:57.463776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:39219)
0808 15:05:57.464233 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:57.464309 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":224,"threads_started":1}
I20260808 15:05:57.606148 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:57.605456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:42431)
0808 15:05:57.605881 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:57.605976 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":215,"threads_started":1}
I20260808 15:05:58.047358 25824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:58.046618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:37815)
0808 15:05:58.047092 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:58.047186 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":232,"threads_started":1}
I20260808 15:05:58.465955 25825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:58.465263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:54155)
0808 15:05:58.465721 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:58.465808 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":197,"threads_started":1}
I20260808 15:05:58.607539 25827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:58.606900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:49155)
0808 15:05:58.607299 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:58.607377 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":214,"threads_started":1}
I20260808 15:05:59.048833 25828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:59.048152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:46951)
0808 15:05:59.048578 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:59.048657 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":230,"threads_started":1}
I20260808 15:05:59.467475 25829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:59.466781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:59753)
0808 15:05:59.467231 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:59.467310 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":241,"threads_started":1}
I20260808 15:05:59.609012 25830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:05:59.608316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:40721)
0808 15:05:59.608749 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0808 15:05:59.608837 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":226,"threads_started":1}
I20260808 15:06:00.050295 25831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:00.049624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:54819)
0808 15:06:00.050039 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:00.050133 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":212,"threads_started":1}
I20260808 15:06:00.468945 25832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:00.468271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:35365)
0808 15:06:00.468706 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:00.468792 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":217,"threads_started":1}
I20260808 15:06:00.610466 25836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:00.609777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:50943)
0808 15:06:00.610191 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:00.610281 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":212,"threads_started":1}
I20260808 15:06:01.051793 25837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:01.051047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:40119)
0808 15:06:01.051490 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:01.051594 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":226,"threads_started":1}
I20260808 15:06:01.470394 25838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:01.469697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:59953)
0808 15:06:01.470112 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:01.470208 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":208,"threads_started":1}
I20260808 15:06:01.612329 25839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:01.611554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.193:41613)
0808 15:06:01.612049 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:01.612144 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":252,"threads_started":1}
I20260808 15:06:01.793947 25840 raft_consensus.cc:482] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:06:01.794137 25840 raft_consensus.cc:504] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12fd35dc0b144c29beced2c53e83d920" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38865 } } peers { permanent_uuid: "dcf60a19737b46148e6d83fb2d818683" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39403 } } peers { permanent_uuid: "0d1659d7ee5a4d45a51c0ae1cec677c6" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38445 } }
I20260808 15:06:01.794813 25840 leader_election.cc:288] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865), dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403)
W20260808 15:06:01.795747 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 12fd35dc0b144c29beced2c53e83d920 (127.2.180.195:38865): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
W20260808 15:06:01.795989 25198 leader_election.cc:341] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer dcf60a19737b46148e6d83fb2d818683 (127.2.180.194:39403): Not found: Tablet not found: d31e3c75698f47959ce83d959fef315f
I20260808 15:06:01.796088 25198 leader_election.cc:302] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d1659d7ee5a4d45a51c0ae1cec677c6; no voters: 12fd35dc0b144c29beced2c53e83d920, dcf60a19737b46148e6d83fb2d818683
I20260808 15:06:01.796308 25840 raft_consensus.cc:2726] T d31e3c75698f47959ce83d959fef315f P 0d1659d7ee5a4d45a51c0ae1cec677c6 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:06:02.053581 25841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:02.052795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.194:40911)
0808 15:06:02.053289 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:02.053417 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":251,"threads_started":1}
I20260808 15:06:02.455394 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 25126
I20260808 15:06:02.469663 25842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:02.456396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44750 (local address 127.2.180.193:38445)
0808 15:06:02.456973 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:02.456980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:02.457009 (+ 29us) server_negotiation.cc:407] Connection header received
0808 15:06:02.457071 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:02.457078 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:02.457347 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:02.457516 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:02.459479 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:02.460527 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:02.461680 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:02.462229 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:02.462321 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:02.464682 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:02.464691 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:02.464705 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:02.464737 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:02.467312 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:02.467962 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:02.467968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:02.467970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:02.468098 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:02.468383 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:02.468387 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:02.468389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:02.468715 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:02.468924 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:02.469141 (+ 217us) server_negotiation.cc:299] Negotiation successful
0808 15:06:02.469347 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":261,"threads_started":1}
I20260808 15:06:02.470821 25361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:02.472355 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:02.471439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:45669 (local address 127.2.180.195:47217)
0808 15:06:02.472004 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:02.472163 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:45669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":249,"threads_started":1}
I20260808 15:06:02.557219 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 25126
I20260808 15:06:02.582649 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 25376
I20260808 15:06:02.595989 25845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:02.583364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.2.180.194:39403)
0808 15:06:02.583928 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:02.583933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:02.583948 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:06:02.584005 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:02.584008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:02.584083 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:02.584182 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:02.585559 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:02.586355 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:02.587643 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:02.588109 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:02.588223 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:02.590695 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:02.590699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:02.590703 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:02.590737 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:02.593378 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:02.594061 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:02.594067 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:02.594071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:02.594214 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:02.594582 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:02.594588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:02.594591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:02.594846 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:02.595096 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:02.595393 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 15:06:02.595596 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":299,"threads_started":1}
I20260808 15:06:02.596695 25561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:02.676896 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 25376
I20260808 15:06:02.699693 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 25573
I20260808 15:06:02.712378 25848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:02.700499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.2.180.195:38865)
0808 15:06:02.701073 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:02.701078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:02.701096 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:06:02.701150 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:02.701154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:02.701224 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:02.701310 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:02.702581 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:02.703348 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:02.704597 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:02.705009 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:02.705114 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:02.707471 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:02.707477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:02.707480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:02.707520 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:02.710160 (+ 2640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:02.710745 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:02.710748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:02.710751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:02.710888 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:02.711197 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:02.711200 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:02.711202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:02.711425 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:02.711651 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:02.711860 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 15:06:02.712027 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":286,"threads_started":1}
I20260808 15:06:02.713045 25756 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:02.794457 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 25573
2026-08-08T15:06:02Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24084 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:06:02 dist-test-slave-jlzn krb5kdc[25852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:06:02 dist-test-slave-jlzn krb5kdc[25852](info): set up 2 sockets
Aug 08 15:06:02 dist-test-slave-jlzn krb5kdc[25852](info): commencing operation
krb5kdc: starting...
W20260808 15:06:03.973881 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.004s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:06:04 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:06:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:06:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:06:04.202735 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46351
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:46351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:06:04.520272 25869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:04.520531 25869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:04.520664 25869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:04.527514 25869 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 15:06:04.527607 25869 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 15:06:04.527670 25869 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 15:06:04.527734 25869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:06:04.527828 25869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:04.527918 25869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:04.527982 25869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:06:04.528054 25869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:06:04.537263 25869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:46351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:04.538336 25869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:04.539521 25869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:04.550563 25879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:04.550705 25884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:04.551184 25878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:04.551733 25869 server_base.cc:1034] running on GCE node
I20260808 15:06:04.552742 25869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:04.554289 25869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:04.555517 25869 hybrid_clock.cc:648] HybridClock initialized: now 1786201564555453 us; error 35 us; skew 500 ppm
I20260808 15:06:04.555886 25869 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:04 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201564, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:04.558795 25869 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:06:04.560114 25869 webserver.cc:460] Webserver started at http://127.2.180.254:38001/ using document root <none> and password file <none>
I20260808 15:06:04.560740 25869 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:04.560839 25869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:04.561116 25869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:06:04.563879 25869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "7dddce20179543428aa99988cee5a142"
format_stamp: "Formatted at 2026-08-08 15:06:04 on dist-test-slave-jlzn"
server_key: "c580af29432c8092b8fe7eedf0474187"
server_key_iv: "d392c5d9339b85ddad6409d490edfa32"
server_key_version: "encryptionkey@0"
I20260808 15:06:04.564612 25869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "7dddce20179543428aa99988cee5a142"
format_stamp: "Formatted at 2026-08-08 15:06:04 on dist-test-slave-jlzn"
server_key: "c580af29432c8092b8fe7eedf0474187"
server_key_iv: "d392c5d9339b85ddad6409d490edfa32"
server_key_version: "encryptionkey@0"
I20260808 15:06:04.569240 25869 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:06:04.572381 25895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:04.573539 25869 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:06:04.573701 25869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "7dddce20179543428aa99988cee5a142"
format_stamp: "Formatted at 2026-08-08 15:06:04 on dist-test-slave-jlzn"
server_key: "c580af29432c8092b8fe7eedf0474187"
server_key_iv: "d392c5d9339b85ddad6409d490edfa32"
server_key_version: "encryptionkey@0"
I20260808 15:06:04.573885 25869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:04.590173 25869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:04.593668 25869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:04.593901 25869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:04.610011 25869 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:46351
I20260808 15:06:04.610016 25959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:46351 every 8 connection(s)
I20260808 15:06:04.611158 25869 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:06:04.611851 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 25869
I20260808 15:06:04.612085 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:06:04.612772 2771 external_mini_cluster.cc:1428] Setting key efaa85036906aab892d454c7da6d6bad
I20260808 15:06:04.615307 25960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 15:06:04 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:06:04.627364 25960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Bootstrap starting.
I20260808 15:06:04.630555 25960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Neither blocks nor log segments found. Creating new log.
I20260808 15:06:04.631745 25960 log.cc:824] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:04.634322 25960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: No bootstrap required, opened a new log
I20260808 15:06:04.635056 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:04.614509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51968 (local address 127.2.180.254:46351)
0808 15:06:04.615219 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:04.615234 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:04.615284 (+ 50us) server_negotiation.cc:407] Connection header received
0808 15:06:04.617493 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:04.617519 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:04.617970 (+ 451us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:04.618395 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:04.619656 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:04.621661 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:04.622945 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:04.623957 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:04.624132 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:04.628145 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:04.628159 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:04.628173 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:04.628205 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:04.631001 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:04.631847 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:04.631854 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:04.631860 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:04.632033 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:04.632480 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:04.632489 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:04.632492 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:04.632987 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:04.633241 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:04.633716 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 15:06:04.634074 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":285,"threads_started":1}
I20260808 15:06:04.641942 25960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:04.642236 25960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:06:04.642330 25960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dddce20179543428aa99988cee5a142, State: Initialized, Role: FOLLOWER
I20260808 15:06:04.642855 25960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:04.643071 25960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:06:04.643201 25960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:06:04.643352 25960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:06:04.645459 25960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:04.645896 25960 leader_election.cc:302] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dddce20179543428aa99988cee5a142; no voters:
I20260808 15:06:04.646413 25960 leader_election.cc:288] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:06:04.646526 25965 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:06:04.647153 25965 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 1 LEADER]: Becoming Leader. State: Replica: 7dddce20179543428aa99988cee5a142, State: Running, Role: LEADER
I20260808 15:06:04.647742 25965 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:04.648139 25960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:06:04.650230 25966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dddce20179543428aa99988cee5a142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } } }
I20260808 15:06:04.650674 25966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: This master's current role is: LEADER
I20260808 15:06:04.651245 25967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dddce20179543428aa99988cee5a142. Latest consensus state: current_term: 1 leader_uuid: "7dddce20179543428aa99988cee5a142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } } }
I20260808 15:06:04.651556 25967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: This master's current role is: LEADER
I20260808 15:06:04.651826 25977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:06:04.656840 25977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:06:04.664595 25977 catalog_manager.cc:1347] Generated new cluster ID: ba4cdf1f85e6405584b737a66a7b2adf
I20260808 15:06:04.664690 25977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:06:04.685105 25977 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:06:04.686234 25977 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:06:04.696975 25977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Generated new TSK 0
I20260808 15:06:04.697703 25977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:06:04.763224 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46351
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:06:05.101306 25996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:05.101562 25996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:05.101696 25996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:05.108476 25996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:05.108579 25996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:05.108731 25996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:06:05.117764 25996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:05.118876 25996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:05.120126 25996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:05.132072 26007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:05.132148 26003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:05.132119 26001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:05.133267 25996 server_base.cc:1034] running on GCE node
I20260808 15:06:05.133836 25996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:05.134670 25996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:05.135877 25996 hybrid_clock.cc:648] HybridClock initialized: now 1786201565135828 us; error 38 us; skew 500 ppm
I20260808 15:06:05.136168 25996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:05 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201565, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:05.139056 25996 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:06:05.140370 25996 webserver.cc:460] Webserver started at http://127.2.180.193:35125/ using document root <none> and password file <none>
I20260808 15:06:05.141008 25996 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:05.141131 25996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:05.141423 25996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:06:05.144207 25996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "b5428147c9ba4b15ac2bcb8585f99562"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "1e38634394653c0408a835c144aa2e63"
server_key_iv: "c49c0638b21fc01b51cc77030ad19c53"
server_key_version: "encryptionkey@0"
I20260808 15:06:05.144922 25996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "b5428147c9ba4b15ac2bcb8585f99562"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "1e38634394653c0408a835c144aa2e63"
server_key_iv: "c49c0638b21fc01b51cc77030ad19c53"
server_key_version: "encryptionkey@0"
I20260808 15:06:05.149801 25996 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 15:06:05.153052 26016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:05.154193 25996 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20260808 15:06:05.154335 25996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "b5428147c9ba4b15ac2bcb8585f99562"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "1e38634394653c0408a835c144aa2e63"
server_key_iv: "c49c0638b21fc01b51cc77030ad19c53"
server_key_version: "encryptionkey@0"
I20260808 15:06:05.154512 25996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:05.179293 25996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:05.182778 25996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:05.183027 25996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:05.183880 25996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:05.185164 25996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:06:05.185243 25996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:05.185357 25996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:06:05.185410 25996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:05.214340 25996 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:34495
I20260808 15:06:05.214395 26171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:34495 every 8 connection(s)
I20260808 15:06:05.215454 25996 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:06:05.216845 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 25996
I20260808 15:06:05.217048 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:06:05.217765 2771 external_mini_cluster.cc:1428] Setting key 34124969be4f162e22821feb6e800449
Aug 08 15:06:05 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201565, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:06:05.233938 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:05.217759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:50637 (local address 127.2.180.254:46351)
0808 15:06:05.219600 (+ 1841us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:05.219607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:05.219627 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:06:05.220949 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:05.220955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:05.221027 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:05.221119 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:05.222475 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:05.223181 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:05.224713 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:05.225096 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:05.225217 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:05.228348 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:05.228354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:05.228358 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:05.228394 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:05.231050 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:05.231720 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:05.231725 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:05.231727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:05.231890 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:05.232311 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:05.232317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:05.232320 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:05.232576 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:05.232719 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:05.233349 (+ 630us) server_negotiation.cc:299] Negotiation successful
0808 15:06:05.233575 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1719,"thread_start_us":1609,"threads_started":1}
I20260808 15:06:05.234736 26176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:05.218119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:50637)
0808 15:06:05.218873 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:05.218927 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:05.220698 (+ 1771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:05.221343 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:05.221351 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:05.221814 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:06:05.222320 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:05.222336 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:05.223316 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:05.223319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:05.224572 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:05.224589 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:05.225305 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:05.225313 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:05.225525 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:05.226095 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:06:05.226116 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:06:05.228134 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:06:05.231187 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:05.231194 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:05.231203 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:05.231592 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:05.231996 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:05.231999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:05.232002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:05.232195 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:05.232695 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:06:05.232703 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:06:05.232985 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 15:06:05.233205 (+ 220us) client_negotiation.cc:240] Negotiation successful
0808 15:06:05.233644 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":299,"threads_started":1}
I20260808 15:06:05.236065 26172 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46351
I20260808 15:06:05.236325 26172 heartbeater.cc:464] Registering TS with master...
I20260808 15:06:05.236953 26172 heartbeater.cc:511] Master 127.2.180.254:46351 requested a full tablet report, sending...
I20260808 15:06:05.238842 25916 ts_manager.cc:194] Registered new tserver with Master: b5428147c9ba4b15ac2bcb8585f99562 (127.2.180.193:34495)
I20260808 15:06:05.240278 25916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:50637
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:06:05.271777 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46351
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:06:05.582010 26181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:05.582252 26181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:05.582350 26181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:05.589222 26181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:05.589315 26181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:05.589427 26181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:06:05.598812 26181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:05.599891 26181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:05.601076 26181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:05.613813 26187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:05.613770 26190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:05.614334 26186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:05.614576 26181 server_base.cc:1034] running on GCE node
I20260808 15:06:05.615224 26181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:05.616295 26181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:05.617512 26181 hybrid_clock.cc:648] HybridClock initialized: now 1786201565617439 us; error 68 us; skew 500 ppm
I20260808 15:06:05.617795 26181 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:05 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201565, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:05.621189 26181 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:06:05.622829 26181 webserver.cc:460] Webserver started at http://127.2.180.194:41937/ using document root <none> and password file <none>
I20260808 15:06:05.623626 26181 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:05.623750 26181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:05.624095 26181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:06:05.627436 26181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "489d2692462746a1a98ed5ed71881b34"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "bbcb43966076d7702c0cf047a29fcd60"
server_key_iv: "bdf956185cb54637a03bf1b874aca6c6"
server_key_version: "encryptionkey@0"
I20260808 15:06:05.628165 26181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "489d2692462746a1a98ed5ed71881b34"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "bbcb43966076d7702c0cf047a29fcd60"
server_key_iv: "bdf956185cb54637a03bf1b874aca6c6"
server_key_version: "encryptionkey@0"
I20260808 15:06:05.632877 26181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:06:05.636075 26199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:05.637244 26181 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:06:05.637367 26181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "489d2692462746a1a98ed5ed71881b34"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "bbcb43966076d7702c0cf047a29fcd60"
server_key_iv: "bdf956185cb54637a03bf1b874aca6c6"
server_key_version: "encryptionkey@0"
I20260808 15:06:05.637548 26181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:05.654536 26181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:05.658311 26181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:05.658591 26181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:05.659433 26181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:05.660805 26181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:06:05.660880 26181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:05.660970 26181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:06:05.661013 26181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:05.689796 26181 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:41341
I20260808 15:06:05.689855 26356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:41341 every 8 connection(s)
I20260808 15:06:05.691032 26181 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:06:05.701050 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 26181
I20260808 15:06:05.701246 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:06:05.701690 2771 external_mini_cluster.cc:1428] Setting key 91e169bc4a5cfd5a0626da6d88b5e74a
Aug 08 15:06:05 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201565, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:06:05.710282 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:05.693408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:38973 (local address 127.2.180.254:46351)
0808 15:06:05.693585 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:05.693589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:05.694651 (+ 1062us) server_negotiation.cc:407] Connection header received
0808 15:06:05.696463 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:05.696470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:05.696548 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:05.696644 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:05.698052 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:05.698772 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:05.700340 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:05.700824 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:05.700951 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:05.704605 (+ 3654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:05.704611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:05.704616 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:05.704652 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:05.707476 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:05.708183 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:05.708188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:05.708191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:05.708340 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:05.708714 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:05.708718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:05.708720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:05.709035 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:05.709178 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:05.709751 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 15:06:05.709973 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:06:05.710894 26360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:05.693824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:38973)
0808 15:06:05.694499 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:05.694543 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:05.696204 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:05.696874 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:05.696882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:05.697377 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:06:05.697889 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:05.697906 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:05.698919 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:05.698923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:05.700189 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:05.700203 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:05.701056 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:05.701061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:05.701309 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:05.701950 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:06:05.701976 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:06:05.704377 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:06:05.707604 (+ 3227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:05.707611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:05.707621 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:05.708063 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:05.708430 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:05.708433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:05.708435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:05.708610 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:05.709138 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:06:05.709143 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:06:05.709405 (+ 262us) client_negotiation.cc:769] Sending connection context
0808 15:06:05.709617 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 15:06:05.709939 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":269,"threads_started":1}
I20260808 15:06:05.712136 26358 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46351
I20260808 15:06:05.712386 26358 heartbeater.cc:464] Registering TS with master...
I20260808 15:06:05.713034 26358 heartbeater.cc:511] Master 127.2.180.254:46351 requested a full tablet report, sending...
I20260808 15:06:05.714401 25916 ts_manager.cc:194] Registered new tserver with Master: 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341)
I20260808 15:06:05.715310 25916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:38973
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:06:05.755611 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46351
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:06:06.067480 26363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:06.067751 26363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:06.067901 26363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:06.074869 26363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:06.074975 26363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:06.075124 26363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:06:06.084327 26363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:06.085433 26363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:06.086661 26363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:06.098742 26376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:06.098780 26374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:06.098814 26373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:06.099521 26363 server_base.cc:1034] running on GCE node
I20260808 15:06:06.100195 26363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:06.101054 26363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:06.102263 26363 hybrid_clock.cc:648] HybridClock initialized: now 1786201566102223 us; error 29 us; skew 500 ppm
I20260808 15:06:06.102546 26363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:06 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201566, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:06.105593 26363 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:06:06.106890 26363 webserver.cc:460] Webserver started at http://127.2.180.195:43021/ using document root <none> and password file <none>
I20260808 15:06:06.107502 26363 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:06.107614 26363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:06.107951 26363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:06:06.110613 26363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "ec9fb1844d67422d9882744235896c9e"
format_stamp: "Formatted at 2026-08-08 15:06:06 on dist-test-slave-jlzn"
server_key: "924f7b05719df33f59eb7134968d8d2a"
server_key_iv: "16640fb457ffe8401401cc1a095f9107"
server_key_version: "encryptionkey@0"
I20260808 15:06:06.111341 26363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "ec9fb1844d67422d9882744235896c9e"
format_stamp: "Formatted at 2026-08-08 15:06:06 on dist-test-slave-jlzn"
server_key: "924f7b05719df33f59eb7134968d8d2a"
server_key_iv: "16640fb457ffe8401401cc1a095f9107"
server_key_version: "encryptionkey@0"
I20260808 15:06:06.116122 26363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 15:06:06.119383 26385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:06.120592 26363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:06:06.120740 26363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "ec9fb1844d67422d9882744235896c9e"
format_stamp: "Formatted at 2026-08-08 15:06:06 on dist-test-slave-jlzn"
server_key: "924f7b05719df33f59eb7134968d8d2a"
server_key_iv: "16640fb457ffe8401401cc1a095f9107"
server_key_version: "encryptionkey@0"
I20260808 15:06:06.120919 26363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:06.139346 26363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:06.143268 26363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:06.143528 26363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:06.144423 26363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:06.145630 26363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:06:06.145701 26363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:06.145817 26363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:06:06.145866 26363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:06.175398 26363 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:46169
I20260808 15:06:06.175438 26537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:46169 every 8 connection(s)
I20260808 15:06:06.176540 26363 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:06:06.184772 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 26363
I20260808 15:06:06.184952 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:06:06.185495 2771 external_mini_cluster.cc:1428] Setting key b865512f5bb7d91573c15b1ebca7a700
Aug 08 15:06:06 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201566, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:06:06.195619 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.178789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:44773 (local address 127.2.180.254:46351)
0808 15:06:06.178994 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.179001 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.180093 (+ 1092us) server_negotiation.cc:407] Connection header received
0808 15:06:06.181776 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.181781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.181861 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.181945 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:06.183236 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.184001 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.186104 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.186614 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.186719 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.190260 (+ 3541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:06.190264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:06.190267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:06.190298 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:06.192824 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.193473 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.193479 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.193482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.193613 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.194008 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.194014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.194016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.194279 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:06.194416 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.195049 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.195278 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:06:06.196436 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.179231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:44773)
0808 15:06:06.179942 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:06.179985 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:06.181553 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:06.182130 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:06.182137 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:06.182590 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:06:06.183110 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.183127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.184151 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.184155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.185942 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.185957 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.186840 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.186848 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.187124 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.187747 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:06:06.187780 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:06:06.190069 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:06:06.192960 (+ 2891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:06.192968 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:06.192982 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:06.193361 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:06.193718 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:06.193723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:06.193727 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:06.193902 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:06.194409 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:06:06.194416 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:06:06.194708 (+ 292us) client_negotiation.cc:769] Sending connection context
0808 15:06:06.194937 (+ 229us) client_negotiation.cc:240] Negotiation successful
0808 15:06:06.195378 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":259,"threads_started":1}
I20260808 15:06:06.197585 26538 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46351
I20260808 15:06:06.197840 26538 heartbeater.cc:464] Registering TS with master...
I20260808 15:06:06.198477 26538 heartbeater.cc:511] Master 127.2.180.254:46351 requested a full tablet report, sending...
I20260808 15:06:06.199831 25915 ts_manager.cc:194] Registered new tserver with Master: ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169)
I20260808 15:06:06.200621 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:06:06.200799 25915 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:44773
I20260808 15:06:06.217002 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.205466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.2.180.254:46351)
0808 15:06:06.205702 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.205710 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.206066 (+ 356us) server_negotiation.cc:407] Connection header received
0808 15:06:06.206144 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.206147 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.206199 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.206271 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:06.207555 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.208273 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.209642 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.210007 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.210094 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.212173 (+ 2079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:06.212178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:06.212181 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:06.212209 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:06.214511 (+ 2302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.215159 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.215163 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.215166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.215287 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.215662 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.215667 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.215669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.216008 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:06.216249 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.216534 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.216725 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:06:06.220597 25915 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:06:06.227284 25915 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:06:06.242808 26172 heartbeater.cc:503] Master 127.2.180.254:46351 was elected leader, sending a full tablet report...
I20260808 15:06:06.264436 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.238703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:34450)
0808 15:06:06.256196 (+ 17493us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:06.256214 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:06.256316 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:06.257316 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:06.257323 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:06.257352 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:06.257808 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.257824 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.260064 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.260069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.261689 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.261706 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.263453 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.263459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.263682 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.263734 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 15:06:06.263869 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 15:06:06.264044 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17352,"mutex_wait_us":3,"thread_start_us":18845,"threads_started":1}
I20260808 15:06:06.265098 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.239351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34450 (local address 127.2.180.193:34495)
0808 15:06:06.256801 (+ 17450us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.256807 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.256827 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:06:06.256892 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.256897 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.257085 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.257241 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:06.257998 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.259954 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.261849 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.263263 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.263382 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.263603 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.264656 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.264808 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17323,"thread_start_us":17196,"threads_started":1}
I20260808 15:06:06.265681 26555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.238218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41341 (local address 127.0.0.1:33866)
0808 15:06:06.254098 (+ 15880us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:06.254134 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:06.254308 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:06.258782 (+ 4474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:06.258787 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:06.258813 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:06.259339 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.259351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.261201 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.261206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.263117 (+ 1911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.263128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.264909 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.264915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.265059 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.265096 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 15:06:06.265167 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 15:06:06.265284 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15658,"mutex_wait_us":20,"thread_start_us":16232,"threads_started":1}
I20260808 15:06:06.265959 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.239379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44360 (local address 127.2.180.195:46169)
0808 15:06:06.255562 (+ 16183us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.255571 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.257932 (+ 2361us) server_negotiation.cc:407] Connection header received
0808 15:06:06.258209 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.258216 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.258415 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.258572 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:06.259932 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.261355 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.263714 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.264688 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.264812 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.265008 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.265516 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.265671 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16025,"thread_start_us":15893,"threads_started":1}
I20260808 15:06:06.266142 26557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.238909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:46169 (local address 127.0.0.1:44360)
0808 15:06:06.257826 (+ 18917us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:06.257843 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:06.258028 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:06.258867 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:06.258876 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:06.258902 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:06.259725 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.259735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.261480 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.261485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.263578 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.263594 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.265137 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.265141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.265300 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.265332 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:06:06.265431 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 15:06:06.265557 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18795,"mutex_wait_us":1,"thread_start_us":16810,"threads_started":1}
I20260808 15:06:06.266812 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.238407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33866 (local address 127.2.180.194:41341)
0808 15:06:06.258213 (+ 19806us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.258220 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.258240 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:06:06.258326 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.258332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.258504 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.259175 (+ 671us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:06.259511 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.261054 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.263260 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.264695 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.265922 (+ 1227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.266097 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.266349 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.266498 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19677,"thread_start_us":19545,"threads_started":1}
I20260808 15:06:06.268916 26083 tablet_service.cc:1467] Processing CreateTablet for tablet e81a2fdd24f647fc86e4c8815aca9770 (DEFAULT_TABLE table=test-table [id=2ef5c7cdb71e45ab9cabc18f581ca485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:06:06.269402 26453 tablet_service.cc:1467] Processing CreateTablet for tablet e81a2fdd24f647fc86e4c8815aca9770 (DEFAULT_TABLE table=test-table [id=2ef5c7cdb71e45ab9cabc18f581ca485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:06:06.270464 26083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81a2fdd24f647fc86e4c8815aca9770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:06:06.270515 26453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81a2fdd24f647fc86e4c8815aca9770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:06:06.270102 26266 tablet_service.cc:1467] Processing CreateTablet for tablet e81a2fdd24f647fc86e4c8815aca9770 (DEFAULT_TABLE table=test-table [id=2ef5c7cdb71e45ab9cabc18f581ca485]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:06:06.271384 26266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e81a2fdd24f647fc86e4c8815aca9770. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:06:06.281173 26561 tablet_bootstrap.cc:492] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Bootstrap starting.
I20260808 15:06:06.283902 26561 tablet_bootstrap.cc:654] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Neither blocks nor log segments found. Creating new log.
I20260808 15:06:06.285172 26562 tablet_bootstrap.cc:492] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: Bootstrap starting.
I20260808 15:06:06.285234 26561 log.cc:824] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:06.286780 26563 tablet_bootstrap.cc:492] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: Bootstrap starting.
I20260808 15:06:06.288327 26561 tablet_bootstrap.cc:492] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: No bootstrap required, opened a new log
I20260808 15:06:06.288576 26561 ts_tablet_manager.cc:1397] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260808 15:06:06.288725 26562 tablet_bootstrap.cc:654] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: Neither blocks nor log segments found. Creating new log.
I20260808 15:06:06.289898 26562 log.cc:824] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:06.289994 26563 tablet_bootstrap.cc:654] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: Neither blocks nor log segments found. Creating new log.
I20260808 15:06:06.291013 26563 log.cc:824] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:06.292802 26562 tablet_bootstrap.cc:492] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: No bootstrap required, opened a new log
I20260808 15:06:06.293098 26562 ts_tablet_manager.cc:1397] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260808 15:06:06.294003 26563 tablet_bootstrap.cc:492] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: No bootstrap required, opened a new log
I20260808 15:06:06.294241 26563 ts_tablet_manager.cc:1397] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 15:06:06.297967 26561 raft_consensus.cc:348] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.298374 26561 raft_consensus.cc:374] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:06:06.298475 26561 raft_consensus.cc:729] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec9fb1844d67422d9882744235896c9e, State: Initialized, Role: FOLLOWER
I20260808 15:06:06.299047 26561 consensus_queue.cc:260] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.300196 26538 heartbeater.cc:503] Master 127.2.180.254:46351 was elected leader, sending a full tablet report...
I20260808 15:06:06.300793 26561 ts_tablet_manager.cc:1428] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 15:06:06.301553 26563 raft_consensus.cc:348] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.301918 26563 raft_consensus.cc:374] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:06:06.302017 26563 raft_consensus.cc:729] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489d2692462746a1a98ed5ed71881b34, State: Initialized, Role: FOLLOWER
I20260808 15:06:06.302525 26563 consensus_queue.cc:260] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.302620 26562 raft_consensus.cc:348] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.303088 26562 raft_consensus.cc:374] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:06:06.303236 26562 raft_consensus.cc:729] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5428147c9ba4b15ac2bcb8585f99562, State: Initialized, Role: FOLLOWER
I20260808 15:06:06.303555 26358 heartbeater.cc:503] Master 127.2.180.254:46351 was elected leader, sending a full tablet report...
I20260808 15:06:06.303879 26562 consensus_queue.cc:260] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.305408 26562 ts_tablet_manager.cc:1428] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260808 15:06:06.307299 26563 ts_tablet_manager.cc:1428] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: Time spent starting tablet: real 0.013s user 0.009s sys 0.000s
I20260808 15:06:06.368449 26571 raft_consensus.cc:482] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:06:06.368688 26571 raft_consensus.cc:504] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.370240 26571 leader_election.cc:288] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341), b5428147c9ba4b15ac2bcb8585f99562 (127.2.180.193:34495)
I20260808 15:06:06.375391 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.370693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:33607 (local address 127.2.180.193:34495)
0808 15:06:06.370895 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.370901 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.370918 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:06:06.371025 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.371029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.371082 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.371160 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:06.371584 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.372294 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.373817 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.374582 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.374731 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.374801 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.375044 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.375162 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 15:06:06.375391 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.370665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.2.180.195:33607)
0808 15:06:06.370818 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:06.370836 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:06.370925 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:06.371217 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:06.371223 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:06.371254 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:06.371429 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.371443 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.372443 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.372447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.373663 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.373677 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.374726 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.374732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.374898 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.374922 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:06:06.375002 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 15:06:06.375128 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260808 15:06:06.376441 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.370670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41341 (local address 127.2.180.195:48499)
0808 15:06:06.371654 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:06.371669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:06.371781 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:06.372088 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:06.372091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:06.372115 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:06.372433 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.372443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.373399 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.373403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.374662 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:06.374674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.375797 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.375830 (+ 33us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:06.375986 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.376010 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:06:06.376082 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 15:06:06.376169 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":888,"mutex_wait_us":326,"thread_start_us":357,"threads_started":1}
I20260808 15:06:06.376525 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.370524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:48499 (local address 127.2.180.194:41341)
0808 15:06:06.370726 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.370732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.371754 (+ 1022us) server_negotiation.cc:407] Connection header received
0808 15:06:06.371923 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.371927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.371984 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.372056 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:06.372599 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.373256 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.374813 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.375613 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.375750 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.375823 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.376140 (+ 317us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.376265 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:06:06.376386 26107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81a2fdd24f647fc86e4c8815aca9770" candidate_uuid: "ec9fb1844d67422d9882744235896c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5428147c9ba4b15ac2bcb8585f99562" is_pre_election: true
I20260808 15:06:06.377000 26107 raft_consensus.cc:2454] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec9fb1844d67422d9882744235896c9e in term 0.
I20260808 15:06:06.377526 26292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81a2fdd24f647fc86e4c8815aca9770" candidate_uuid: "ec9fb1844d67422d9882744235896c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489d2692462746a1a98ed5ed71881b34" is_pre_election: true
I20260808 15:06:06.377668 26389 leader_election.cc:302] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5428147c9ba4b15ac2bcb8585f99562, ec9fb1844d67422d9882744235896c9e; no voters:
I20260808 15:06:06.377926 26292 raft_consensus.cc:2454] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ec9fb1844d67422d9882744235896c9e in term 0.
I20260808 15:06:06.378000 26571 raft_consensus.cc:2781] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:06:06.378137 26571 raft_consensus.cc:482] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:06:06.378235 26571 raft_consensus.cc:3037] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:06:06.380421 26571 raft_consensus.cc:504] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.380968 26571 leader_election.cc:288] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [CANDIDATE]: Term 1 election: Requested vote from peers 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341), b5428147c9ba4b15ac2bcb8585f99562 (127.2.180.193:34495)
I20260808 15:06:06.381336 26292 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81a2fdd24f647fc86e4c8815aca9770" candidate_uuid: "ec9fb1844d67422d9882744235896c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489d2692462746a1a98ed5ed71881b34"
I20260808 15:06:06.381392 26107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e81a2fdd24f647fc86e4c8815aca9770" candidate_uuid: "ec9fb1844d67422d9882744235896c9e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5428147c9ba4b15ac2bcb8585f99562"
I20260808 15:06:06.381494 26292 raft_consensus.cc:3037] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:06:06.381560 26107 raft_consensus.cc:3037] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:06:06.383569 26292 raft_consensus.cc:2454] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec9fb1844d67422d9882744235896c9e in term 1.
I20260808 15:06:06.383603 26107 raft_consensus.cc:2454] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ec9fb1844d67422d9882744235896c9e in term 1.
I20260808 15:06:06.383940 26388 leader_election.cc:302] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 489d2692462746a1a98ed5ed71881b34, ec9fb1844d67422d9882744235896c9e; no voters:
I20260808 15:06:06.384255 26571 raft_consensus.cc:2781] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:06:06.384662 26571 raft_consensus.cc:686] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 1 LEADER]: Becoming Leader. State: Replica: ec9fb1844d67422d9882744235896c9e, State: Running, Role: LEADER
I20260808 15:06:06.385128 26571 consensus_queue.cc:237] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } }
I20260808 15:06:06.389173 25916 catalog_manager.cc:5495] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e reported cstate change: term changed from 0 to 1, leader changed from <none> to ec9fb1844d67422d9882744235896c9e (127.2.180.195). New cstate: current_term: 1 leader_uuid: "ec9fb1844d67422d9882744235896c9e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } health_report { overall_health: HEALTHY } } }
I20260808 15:06:06.417102 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.412493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44372 (local address 127.2.180.195:46169)
0808 15:06:06.412667 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.412672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.412688 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:06:06.412849 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.412852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.412921 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.413032 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:06:06.413336 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.413994 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.415312 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.415683 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.415766 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.416185 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:06:06.416636 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:06:06.416746 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.416782 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.416840 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:06:06.424610 26292 raft_consensus.cc:1261] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 1 FOLLOWER]: Refusing update from remote peer ec9fb1844d67422d9882744235896c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:06:06.424621 26107 raft_consensus.cc:1261] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Refusing update from remote peer ec9fb1844d67422d9882744235896c9e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:06:06.425549 26575 consensus_queue.cc:1035] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:06:06.425930 26571 consensus_queue.cc:1035] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:06:06.428198 26539 tablet.cc:2369] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:06:06.436204 26581 mvcc.cc:205] Tried to move back new op lower bound from 7316281616065622016 to 7316281615917023232. Current Snapshot: MvccSnapshot[applied={T|T < 7316281616065622016}]
I20260808 15:06:06.448619 25916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:06:06.448854 25916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:06:06.452273 25916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Sending DeleteTablet for 3 replicas of tablet e81a2fdd24f647fc86e4c8815aca9770
I20260808 15:06:06.453299 26266 tablet_service.cc:1514] Processing DeleteTablet for tablet e81a2fdd24f647fc86e4c8815aca9770 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:06:06 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:33866
I20260808 15:06:06.453332 26453 tablet_service.cc:1514] Processing DeleteTablet for tablet e81a2fdd24f647fc86e4c8815aca9770 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:06:06 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:44360
I20260808 15:06:06.453356 26083 tablet_service.cc:1514] Processing DeleteTablet for tablet e81a2fdd24f647fc86e4c8815aca9770 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:06:06 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:34450
I20260808 15:06:06.460006 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 25996
I20260808 15:06:06.465508 26594 tablet_replica.cc:330] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: stopping tablet replica
I20260808 15:06:06.465988 26594 raft_consensus.cc:2229] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:06:06.466506 26594 raft_consensus.cc:2258] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:06:06.468430 26594 ts_tablet_manager.cc:1905] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:06:06.469677 26598 tablet_replica.cc:330] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: stopping tablet replica
I20260808 15:06:06.470137 26598 raft_consensus.cc:2229] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:06:06.470624 26598 raft_consensus.cc:2258] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:06:06.472400 26598 ts_tablet_manager.cc:1905] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:06:06.475363 26594 ts_tablet_manager.cc:1918] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:06:06.475567 26594 log.cc:1196] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/e81a2fdd24f647fc86e4c8815aca9770
I20260808 15:06:06.476236 26594 ts_tablet_manager.cc:1939] T e81a2fdd24f647fc86e4c8815aca9770 P 489d2692462746a1a98ed5ed71881b34: Deleting consensus metadata
I20260808 15:06:06.478040 25897 catalog_manager.cc:4841] TS 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341): tablet e81a2fdd24f647fc86e4c8815aca9770 (table test-table [id=2ef5c7cdb71e45ab9cabc18f581ca485]) successfully deleted
I20260808 15:06:06.479338 26596 tablet_replica.cc:330] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: stopping tablet replica
I20260808 15:06:06.479312 26598 ts_tablet_manager.cc:1918] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:06:06.479483 26598 log.cc:1196] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/e81a2fdd24f647fc86e4c8815aca9770
I20260808 15:06:06.479897 26596 raft_consensus.cc:2229] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:06:06.480137 26598 ts_tablet_manager.cc:1939] T e81a2fdd24f647fc86e4c8815aca9770 P b5428147c9ba4b15ac2bcb8585f99562: Deleting consensus metadata
I20260808 15:06:06.480551 26596 raft_consensus.cc:2258] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:06:06.481803 25899 catalog_manager.cc:4841] TS b5428147c9ba4b15ac2bcb8585f99562 (127.2.180.193:34495): tablet e81a2fdd24f647fc86e4c8815aca9770 (table test-table [id=2ef5c7cdb71e45ab9cabc18f581ca485]) successfully deleted
I20260808 15:06:06.482520 26596 ts_tablet_manager.cc:1905] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 15:06:06 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:06:06.488895 26596 ts_tablet_manager.cc:1918] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:06:06.489050 26596 log.cc:1196] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/e81a2fdd24f647fc86e4c8815aca9770
I20260808 15:06:06.489468 26596 ts_tablet_manager.cc:1939] T e81a2fdd24f647fc86e4c8815aca9770 P ec9fb1844d67422d9882744235896c9e: Deleting consensus metadata
I20260808 15:06:06.490821 25896 catalog_manager.cc:4841] TS ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169): tablet e81a2fdd24f647fc86e4c8815aca9770 (table test-table [id=2ef5c7cdb71e45ab9cabc18f581ca485]) successfully deleted
I20260808 15:06:06.494401 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.461126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.2.180.193:34495)
0808 15:06:06.461295 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.461302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.475596 (+ 14294us) server_negotiation.cc:407] Connection header received
0808 15:06:06.476081 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.476087 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.476167 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.476245 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:06.478526 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.479668 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.481684 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.482281 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.482376 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.487711 (+ 5335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:06.487726 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:06.487744 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:06.487783 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:06.491564 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.492341 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.492350 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.492354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.492514 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.492858 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.492863 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.492866 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.493336 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:06.493653 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.493824 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.494040 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:06:06.494997 26161 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:06.583266 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 25996
I20260808 15:06:06.606356 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 26181
Aug 08 15:06:06 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:06:06.625600 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.607184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33872 (local address 127.2.180.194:41341)
0808 15:06:06.607366 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.607372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.607393 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:06:06.607547 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.607552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.607622 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.607707 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:06.609141 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.610226 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.612274 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.612895 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.613029 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.618521 (+ 5492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:06.618542 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:06.618563 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:06.618608 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:06.622273 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.623072 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.623082 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.623086 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.623244 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.623673 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.623678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.623681 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.624303 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:06.624592 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.624943 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.625252 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:06:06.626323 26351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:06.722527 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 26181
I20260808 15:06:06.745728 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 26363
Aug 08 15:06:06 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:06:06.762833 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:06.746613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44378 (local address 127.2.180.195:46169)
0808 15:06:06.746826 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:06.746832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:06.746855 (+ 23us) server_negotiation.cc:407] Connection header received
0808 15:06:06.747054 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:06.747071 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:06.747160 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:06.747273 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:06.748632 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.749914 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.751298 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:06.752154 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:06.752336 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:06.756763 (+ 4427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:06.756776 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:06.756792 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:06.756832 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:06.759724 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.760574 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.760582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.760584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.760755 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:06.761266 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:06.761270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:06.761272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:06.761707 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:06.761943 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:06.762295 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 15:06:06.762462 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:06:06.763589 26532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:06.886168 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 26363
I20260808 15:06:06.913690 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 25869
I20260808 15:06:06.914740 25954 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:07.021538 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 25869
I20260808 15:06:27.046805 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46351
--webserver_interface=127.2.180.254
--webserver_port=38001
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:46351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:06:27.368549 26605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:27.368820 26605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:27.368957 26605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:27.375771 26605 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 15:06:27.375881 26605 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 15:06:27.375943 26605 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 15:06:27.376010 26605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:06:27.376081 26605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:27.376173 26605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:27.376235 26605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:06:27.376308 26605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:06:27.385432 26605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:46351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=38001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:27.386510 26605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:27.387696 26605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:27.410703 26622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:27.410936 26605 server_base.cc:1034] running on GCE node
W20260808 15:06:27.410774 26615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:27.410737 26618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:27.412024 26605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:27.413431 26605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:27.414644 26605 hybrid_clock.cc:648] HybridClock initialized: now 1786201587414576 us; error 45 us; skew 500 ppm
I20260808 15:06:27.415004 26605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:27 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201587, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:27.422250 26605 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:06:27.423733 26605 webserver.cc:460] Webserver started at http://127.2.180.254:38001/ using document root <none> and password file <none>
I20260808 15:06:27.424521 26605 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:27.424626 26605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:27.430383 26605 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 15:06:27.433190 26629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:27.434425 26605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 15:06:27.434587 26605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "7dddce20179543428aa99988cee5a142"
format_stamp: "Formatted at 2026-08-08 15:06:04 on dist-test-slave-jlzn"
server_key: "c580af29432c8092b8fe7eedf0474187"
server_key_iv: "d392c5d9339b85ddad6409d490edfa32"
server_key_version: "encryptionkey@0"
I20260808 15:06:27.435585 26605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:27.455681 26605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:27.459329 26605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:27.459561 26605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:27.475956 26605 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:46351
I20260808 15:06:27.475978 26698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:46351 every 8 connection(s)
I20260808 15:06:27.477324 26605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:06:27.480886 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 26605
I20260808 15:06:27.481704 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:34495
--local_ip_for_outbound_sockets=127.2.180.193
--tserver_master_addrs=127.2.180.254:46351
--webserver_port=35125
--webserver_interface=127.2.180.193
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:06:27.484207 26699 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:06:27.487345 26699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Bootstrap starting.
I20260808 15:06:27.511684 26699 log.cc:824] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:27.526940 26699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:06:27.527576 26699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Bootstrap complete.
I20260808 15:06:27.540727 26699 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:27.541858 26699 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dddce20179543428aa99988cee5a142, State: Initialized, Role: FOLLOWER
I20260808 15:06:27.542573 26699 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:27.542860 26699 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:06:27.542981 26699 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:06:27.543182 26699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 1 FOLLOWER]: Advancing to term 2
I20260808 15:06:27.548669 26699 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:27.549271 26699 leader_election.cc:302] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dddce20179543428aa99988cee5a142; no voters:
I20260808 15:06:27.549870 26699 leader_election.cc:288] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 15:06:27.549986 26705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 15:06:27.551122 26705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 2 LEADER]: Becoming Leader. State: Replica: 7dddce20179543428aa99988cee5a142, State: Running, Role: LEADER
I20260808 15:06:27.551666 26705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:27.552084 26699 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:06:27.553802 26708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dddce20179543428aa99988cee5a142. Latest consensus state: current_term: 2 leader_uuid: "7dddce20179543428aa99988cee5a142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } } }
I20260808 15:06:27.554150 26708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: This master's current role is: LEADER
I20260808 15:06:27.554678 26706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7dddce20179543428aa99988cee5a142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } } }
I20260808 15:06:27.554988 26706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: This master's current role is: LEADER
I20260808 15:06:27.565297 26712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:06:27.572944 26712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:06:27.574383 26712 catalog_manager.cc:1259] Loaded cluster ID: ba4cdf1f85e6405584b737a66a7b2adf
I20260808 15:06:27.574522 26712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:06:27.578425 26712 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:06:27.579967 26712 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Loaded TSK: 0
I20260808 15:06:27.617614 26712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Generated new TSK 1
I20260808 15:06:27.618047 26712 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:06:27.847247 26703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:27.847519 26703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:27.847702 26703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:27.854796 26703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:27.854905 26703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:27.855047 26703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:06:27.864301 26703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:34495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=35125
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:27.865475 26703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:27.866807 26703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:27.880964 26737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:27.881599 26742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:27.881765 26735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:27.882042 26703 server_base.cc:1034] running on GCE node
I20260808 15:06:27.882615 26703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:27.883483 26703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:27.884680 26703 hybrid_clock.cc:648] HybridClock initialized: now 1786201587884625 us; error 49 us; skew 500 ppm
I20260808 15:06:27.884971 26703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:27 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201587, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:27.887991 26703 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:06:27.889401 26703 webserver.cc:460] Webserver started at http://127.2.180.193:35125/ using document root <none> and password file <none>
I20260808 15:06:27.890044 26703 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:27.890175 26703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:27.895226 26703 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:06:27.897889 26757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:27.899251 26703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 15:06:27.899403 26703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "b5428147c9ba4b15ac2bcb8585f99562"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "1e38634394653c0408a835c144aa2e63"
server_key_iv: "c49c0638b21fc01b51cc77030ad19c53"
server_key_version: "encryptionkey@0"
I20260808 15:06:27.900389 26703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:27.932567 26703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:27.936509 26703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:27.936748 26703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:27.937660 26703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:27.938994 26703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:06:27.939117 26703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:27.939224 26703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:06:27.939294 26703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:27.968950 26703 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:34495
I20260808 15:06:27.969012 26901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:34495 every 8 connection(s)
I20260808 15:06:27.970239 26703 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:06:27.979123 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 26703
I20260808 15:06:27.979868 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:41341
--local_ip_for_outbound_sockets=127.2.180.194
--tserver_master_addrs=127.2.180.254:46351
--webserver_port=41937
--webserver_interface=127.2.180.194
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:06:27 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201587, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:06:27.992311 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:27.972875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:53037 (local address 127.2.180.254:46351)
0808 15:06:27.973402 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:27.973413 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:27.974017 (+ 604us) server_negotiation.cc:407] Connection header received
0808 15:06:27.976046 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:27.976058 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:27.976359 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:27.976630 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:27.977902 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:27.979462 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:27.981075 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:27.981977 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:27.982138 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:27.985139 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:27.985157 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:27.985177 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:27.985222 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:27.988472 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:27.989121 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:27.989128 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:27.989135 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:27.989329 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:27.989694 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:27.989698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:27.989701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:27.990240 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:27.990402 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:27.991130 (+ 728us) server_negotiation.cc:299] Negotiation successful
0808 15:06:27.991420 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":246,"threads_started":1}
I20260808 15:06:27.992636 26905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:27.973058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:53037)
0808 15:06:27.973861 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:27.973913 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:27.975593 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:27.976774 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:27.976781 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:27.977235 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:06:27.977749 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:27.977768 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:27.979625 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:27.979629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:27.980922 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:27.980944 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:27.982156 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:27.982161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:27.982342 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:27.982893 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:06:27.982913 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:06:27.984899 (+ 1986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:06:27.988595 (+ 3696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:27.988602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:27.988611 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:27.988986 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:27.989418 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:27.989421 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:27.989423 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:27.989599 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:27.990371 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:06:27.990378 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:06:27.990730 (+ 352us) client_negotiation.cc:769] Sending connection context
0808 15:06:27.990998 (+ 268us) client_negotiation.cc:240] Negotiation successful
0808 15:06:27.991458 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":327,"threads_started":1}
I20260808 15:06:27.995311 26902 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46351
I20260808 15:06:27.995561 26902 heartbeater.cc:464] Registering TS with master...
I20260808 15:06:27.996243 26902 heartbeater.cc:511] Master 127.2.180.254:46351 requested a full tablet report, sending...
I20260808 15:06:27.998457 26649 ts_manager.cc:194] Registered new tserver with Master: b5428147c9ba4b15ac2bcb8585f99562 (127.2.180.193:34495)
I20260808 15:06:28.000411 26649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:53037
W20260808 15:06:28.303373 26908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:28.303648 26908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:28.303751 26908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:28.310636 26908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:28.310726 26908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:28.310837 26908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:06:28.319907 26908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:41341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=41937
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:28.320992 26908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:28.322162 26908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:28.334395 26914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:28.334334 26916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:28.334779 26913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:28.335320 26908 server_base.cc:1034] running on GCE node
I20260808 15:06:28.335871 26908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:28.336678 26908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:28.337872 26908 hybrid_clock.cc:648] HybridClock initialized: now 1786201588337830 us; error 33 us; skew 500 ppm
I20260808 15:06:28.338160 26908 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:28 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201588, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:28.341257 26908 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:06:28.342624 26908 webserver.cc:460] Webserver started at http://127.2.180.194:41937/ using document root <none> and password file <none>
I20260808 15:06:28.343231 26908 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:28.343364 26908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:28.348297 26908 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:06:28.350723 26926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:28.351927 26908 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:06:28.352079 26908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "489d2692462746a1a98ed5ed71881b34"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "bbcb43966076d7702c0cf047a29fcd60"
server_key_iv: "bdf956185cb54637a03bf1b874aca6c6"
server_key_version: "encryptionkey@0"
I20260808 15:06:28.352998 26908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:28.371117 26908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:28.374943 26908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:28.375180 26908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:28.376039 26908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:28.377239 26908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:06:28.377336 26908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:28.377450 26908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:06:28.377522 26908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:28.407176 26908 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:41341
I20260808 15:06:28.407234 27080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:41341 every 8 connection(s)
I20260808 15:06:28.408541 26908 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:06:28.417959 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 26908
I20260808 15:06:28.418588 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:46169
--local_ip_for_outbound_sockets=127.2.180.195
--tserver_master_addrs=127.2.180.254:46351
--webserver_port=43021
--webserver_interface=127.2.180.195
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 15:06:28 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201588, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:06:28.426702 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.410805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:46221 (local address 127.2.180.254:46351)
0808 15:06:28.410971 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:28.410976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:28.412078 (+ 1102us) server_negotiation.cc:407] Connection header received
0808 15:06:28.413808 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:28.413814 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:28.413894 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:28.413979 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:28.415379 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.416101 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.417625 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.418041 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.418171 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.421354 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:28.421358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:28.421362 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:28.421394 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:28.423760 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:28.424435 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:28.424441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:28.424444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:28.424562 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:28.424995 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:28.425000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:28.425004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:28.425281 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:28.425412 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:28.426098 (+ 686us) server_negotiation.cc:299] Negotiation successful
0808 15:06:28.426344 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 15:06:28.427557 27085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.411231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:46221)
0808 15:06:28.411942 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:28.411978 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:28.413540 (+ 1562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:28.414215 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:28.414223 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:28.414707 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:06:28.415218 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.415234 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.416270 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.416274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.417470 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.417482 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.418266 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.418274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.418501 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.419097 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:06:28.419122 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:06:28.421149 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:06:28.423903 (+ 2754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:28.423912 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:28.423925 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:28.424326 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:28.424671 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:28.424676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:28.424680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:28.424894 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:28.425403 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:06:28.425409 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:06:28.425755 (+ 346us) client_negotiation.cc:769] Sending connection context
0808 15:06:28.426003 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 15:06:28.426467 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":272,"threads_started":1}
I20260808 15:06:28.429028 27082 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46351
I20260808 15:06:28.429297 27082 heartbeater.cc:464] Registering TS with master...
I20260808 15:06:28.429966 27082 heartbeater.cc:511] Master 127.2.180.254:46351 requested a full tablet report, sending...
I20260808 15:06:28.431289 26649 ts_manager.cc:194] Registered new tserver with Master: 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341)
I20260808 15:06:28.432111 26649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:46221
W20260808 15:06:28.740849 27086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:28.741087 27086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:28.741211 27086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:28.748095 27086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:28.748193 27086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:28.748342 27086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:06:28.757611 27086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:46169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=43021
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:28.758698 27086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:28.760078 27086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:28.772334 27092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:28.772415 27096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:28.772384 27099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:28.773094 27086 server_base.cc:1034] running on GCE node
I20260808 15:06:28.773651 27086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:28.774482 27086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:28.775686 27086 hybrid_clock.cc:648] HybridClock initialized: now 1786201588775623 us; error 47 us; skew 500 ppm
I20260808 15:06:28.776007 27086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:28 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201588, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:28.778872 27086 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:06:28.780233 27086 webserver.cc:460] Webserver started at http://127.2.180.195:43021/ using document root <none> and password file <none>
I20260808 15:06:28.780848 27086 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:28.780978 27086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:28.785880 27086 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 15:06:28.788390 27107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:28.789523 27086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:06:28.789675 27086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "ec9fb1844d67422d9882744235896c9e"
format_stamp: "Formatted at 2026-08-08 15:06:06 on dist-test-slave-jlzn"
server_key: "924f7b05719df33f59eb7134968d8d2a"
server_key_iv: "16640fb457ffe8401401cc1a095f9107"
server_key_version: "encryptionkey@0"
I20260808 15:06:28.790572 27086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:28.818522 27086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:28.822144 27086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:28.822391 27086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:28.823311 27086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:28.824707 27086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:06:28.824827 27086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:28.824935 27086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:06:28.825006 27086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:28.855557 27086 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:46169
I20260808 15:06:28.855608 27270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:46169 every 8 connection(s)
I20260808 15:06:28.856977 27086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:06:28.864060 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 27086
Aug 08 15:06:28 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201588, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:06:28.878926 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.859237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:60229 (local address 127.2.180.254:46351)
0808 15:06:28.859441 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:28.859447 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:28.860546 (+ 1099us) server_negotiation.cc:407] Connection header received
0808 15:06:28.862275 (+ 1729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:28.862279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:28.862346 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:28.862428 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:28.864033 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.865194 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.868471 (+ 3277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.869085 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.869196 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.873399 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:28.873403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:28.873406 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:28.873438 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:28.876020 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:28.876668 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:28.876672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:28.876674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:28.876804 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:28.877208 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:28.877212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:28.877214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:28.877490 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:28.877636 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:28.878405 (+ 769us) server_negotiation.cc:299] Negotiation successful
0808 15:06:28.878580 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:06:28.880087 27273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.859684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:60229)
0808 15:06:28.860386 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:28.860430 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:28.862036 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:28.862611 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:28.862617 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:28.863092 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:06:28.863752 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.863779 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.865416 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.865422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.868324 (+ 2902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.868340 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.869358 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.869366 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.869573 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.870375 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:06:28.870404 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:06:28.873200 (+ 2796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:06:28.876141 (+ 2941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:28.876149 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:28.876163 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:28.876548 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:28.876907 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:28.876912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:28.876916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:28.877090 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:28.877634 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:06:28.877642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:06:28.878017 (+ 375us) client_negotiation.cc:769] Sending connection context
0808 15:06:28.878312 (+ 295us) client_negotiation.cc:240] Negotiation successful
0808 15:06:28.878778 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":292,"threads_started":1}
I20260808 15:06:28.881435 27271 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46351
I20260808 15:06:28.881708 27271 heartbeater.cc:464] Registering TS with master...
I20260808 15:06:28.882340 27271 heartbeater.cc:511] Master 127.2.180.254:46351 requested a full tablet report, sending...
I20260808 15:06:28.882827 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.865090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34566 (local address 127.2.180.254:46351)
0808 15:06:28.868837 (+ 3747us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:28.868842 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:28.868861 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:06:28.868910 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:28.868914 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:28.868982 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:28.869085 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:28.871257 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.872435 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.873882 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.874309 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.874423 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.877032 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:28.877036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:28.877039 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:28.877069 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:28.880197 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:28.880958 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:28.880961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:28.880963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:28.881107 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:28.881450 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:28.881455 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:28.881459 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:28.881732 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:28.881970 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:28.882299 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 15:06:28.882495 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3618,"thread_start_us":3518,"threads_started":1}
I20260808 15:06:28.883870 26649 ts_manager.cc:194] Registered new tserver with Master: ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169)
I20260808 15:06:28.884727 26649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:60229
I20260808 15:06:28.885268 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:06:28.892583 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.886338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34582 (local address 127.2.180.254:46351)
0808 15:06:28.886513 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:28.886518 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:28.888310 (+ 1792us) server_negotiation.cc:407] Connection header received
0808 15:06:28.888465 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:28.888468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:28.888606 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:28.888714 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:06:28.889042 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.889687 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.891043 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.891434 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.891537 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.891976 (+ 439us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:06:28.892148 (+ 172us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 15:06:28.892382 (+ 234us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34582: token expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 15:06:28.892683 27275 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34582: token expired
W20260808 15:06:28.893208 26541 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.2.180.254:46351)
I20260808 15:06:28.904465 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.893677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34588 (local address 127.2.180.254:46351)
0808 15:06:28.893828 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:28.893833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:28.893861 (+ 28us) server_negotiation.cc:407] Connection header received
0808 15:06:28.893902 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:28.893905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:28.893952 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:28.894020 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:28.895329 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.896046 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.897280 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.897670 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.897772 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.900121 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:28.900126 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:28.900129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:28.900156 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:28.902175 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:28.902747 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:28.902753 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:28.902756 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:28.902876 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:28.903200 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:28.903203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:28.903205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:28.903517 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:28.903737 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:28.904024 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 15:06:28.904194 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:06:28.907343 26649 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:06:28.914363 26649 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:06:28.950075 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.925371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38222 (local address 127.2.180.193:34495)
0808 15:06:28.941183 (+ 15812us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:28.941192 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:28.943033 (+ 1841us) server_negotiation.cc:407] Connection header received
0808 15:06:28.943136 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:28.943142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:28.943421 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:28.943591 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:28.944475 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.945697 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.947969 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.949010 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.949144 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.949335 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:28.949624 (+ 289us) server_negotiation.cc:299] Negotiation successful
0808 15:06:28.949781 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15639,"thread_start_us":15673,"threads_started":1}
I20260808 15:06:28.950136 27292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.924892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38222)
0808 15:06:28.942693 (+ 17801us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:28.942712 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:28.942842 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:28.943867 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:28.943872 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:28.943896 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:28.944288 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.944309 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.945854 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.945859 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.947791 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.947835 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.949226 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.949233 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.949410 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.949444 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 15:06:28.949546 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 15:06:28.949730 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17674,"mutex_wait_us":2,"thread_start_us":16432,"threads_started":1}
I20260808 15:06:28.950687 27289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.925281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41341 (local address 127.0.0.1:35470)
0808 15:06:28.943030 (+ 17749us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:28.943053 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:28.943190 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:28.944145 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:28.944150 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:28.944173 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:28.944536 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.944549 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.945990 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.945999 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.948227 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.948242 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.949830 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.949837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.950005 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.950041 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:06:28.950130 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 15:06:28.950264 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17632,"mutex_wait_us":28,"thread_start_us":17144,"threads_started":1}
I20260808 15:06:28.951210 27290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.924857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44280 (local address 127.2.180.195:46169)
0808 15:06:28.940275 (+ 15418us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:28.940284 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:28.942108 (+ 1824us) server_negotiation.cc:407] Connection header received
0808 15:06:28.942207 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:28.942215 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:28.942448 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:28.942617 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:28.943620 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.945119 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.947864 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.949012 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.949143 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.949367 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:28.950781 (+ 1414us) server_negotiation.cc:299] Negotiation successful
0808 15:06:28.950935 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15241,"thread_start_us":15437,"threads_started":1}
I20260808 15:06:28.951323 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.925892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.2.180.194:41341)
0808 15:06:28.942248 (+ 16356us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:28.942258 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:28.943315 (+ 1057us) server_negotiation.cc:407] Connection header received
0808 15:06:28.943429 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:28.943436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:28.943697 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:28.943915 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:28.944706 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.945844 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.948387 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.949626 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.949754 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.949922 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:28.950900 (+ 978us) server_negotiation.cc:299] Negotiation successful
0808 15:06:28.951050 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16219,"thread_start_us":16127,"threads_started":1}
I20260808 15:06:28.951628 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:28.924722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:46169 (local address 127.0.0.1:44280)
0808 15:06:28.941711 (+ 16989us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:28.941750 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:28.941925 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:28.942948 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:28.942955 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:28.942990 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:28.943400 (+ 410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.943412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.945257 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.945263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.947673 (+ 2410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:28.947689 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:28.950085 (+ 2396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:28.950091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:28.950272 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:28.950300 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 15:06:28.950388 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 15:06:28.950504 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16751,"mutex_wait_us":46,"thread_start_us":18042,"threads_started":1}
I20260808 15:06:28.953858 26820 tablet_service.cc:1467] Processing CreateTablet for tablet 3134a55adbc14f28a5ea1803fd1a053a (DEFAULT_TABLE table=test-table [id=07b5589a59cd4fc6ac61addf1c8adc06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:06:28.955015 26820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3134a55adbc14f28a5ea1803fd1a053a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:06:28.954986 27184 tablet_service.cc:1467] Processing CreateTablet for tablet 3134a55adbc14f28a5ea1803fd1a053a (DEFAULT_TABLE table=test-table [id=07b5589a59cd4fc6ac61addf1c8adc06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:06:28.954986 26991 tablet_service.cc:1467] Processing CreateTablet for tablet 3134a55adbc14f28a5ea1803fd1a053a (DEFAULT_TABLE table=test-table [id=07b5589a59cd4fc6ac61addf1c8adc06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:06:28.956295 27184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3134a55adbc14f28a5ea1803fd1a053a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:06:28.956333 26991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3134a55adbc14f28a5ea1803fd1a053a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:06:28.966919 27297 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Bootstrap starting.
I20260808 15:06:28.971524 27297 tablet_bootstrap.cc:654] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Neither blocks nor log segments found. Creating new log.
I20260808 15:06:28.972855 27297 log.cc:824] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:28.975603 27301 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: Bootstrap starting.
I20260808 15:06:28.979080 27301 tablet_bootstrap.cc:654] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: Neither blocks nor log segments found. Creating new log.
I20260808 15:06:28.981729 27300 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Bootstrap starting.
I20260808 15:06:28.982936 27301 log.cc:824] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:28.984980 27300 tablet_bootstrap.cc:654] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Neither blocks nor log segments found. Creating new log.
I20260808 15:06:28.986116 27300 log.cc:824] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:28.986559 27301 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: No bootstrap required, opened a new log
I20260808 15:06:28.986836 27301 ts_tablet_manager.cc:1397] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260808 15:06:28.986867 27297 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: No bootstrap required, opened a new log
I20260808 15:06:28.987087 27297 ts_tablet_manager.cc:1397] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Time spent bootstrapping tablet: real 0.020s user 0.013s sys 0.003s
I20260808 15:06:28.989089 27300 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: No bootstrap required, opened a new log
I20260808 15:06:28.989328 27300 ts_tablet_manager.cc:1397] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 15:06:28.994086 27297 raft_consensus.cc:348] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:28.994460 27297 raft_consensus.cc:374] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:06:28.994558 27297 raft_consensus.cc:729] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5428147c9ba4b15ac2bcb8585f99562, State: Initialized, Role: FOLLOWER
I20260808 15:06:28.995111 27297 consensus_queue.cc:260] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:28.996109 26902 heartbeater.cc:503] Master 127.2.180.254:46351 was elected leader, sending a full tablet report...
I20260808 15:06:28.996706 27297 ts_tablet_manager.cc:1428] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Time spent starting tablet: real 0.010s user 0.010s sys 0.001s
I20260808 15:06:28.997344 27301 raft_consensus.cc:348] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:28.997746 27301 raft_consensus.cc:374] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:06:28.997844 27301 raft_consensus.cc:729] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489d2692462746a1a98ed5ed71881b34, State: Initialized, Role: FOLLOWER
I20260808 15:06:28.998394 27301 consensus_queue.cc:260] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:28.999208 27300 raft_consensus.cc:348] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:28.999570 27082 heartbeater.cc:503] Master 127.2.180.254:46351 was elected leader, sending a full tablet report...
I20260808 15:06:28.999601 27300 raft_consensus.cc:374] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:06:28.999696 27300 raft_consensus.cc:729] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec9fb1844d67422d9882744235896c9e, State: Initialized, Role: FOLLOWER
I20260808 15:06:29.000106 27301 ts_tablet_manager.cc:1428] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: Time spent starting tablet: real 0.013s user 0.012s sys 0.002s
I20260808 15:06:29.000252 27300 consensus_queue.cc:260] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:29.001394 27271 heartbeater.cc:503] Master 127.2.180.254:46351 was elected leader, sending a full tablet report...
I20260808 15:06:29.001981 27300 ts_tablet_manager.cc:1428] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
I20260808 15:06:29.066614 27306 raft_consensus.cc:482] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:06:29.066891 27306 raft_consensus.cc:504] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:29.068547 27306 leader_election.cc:288] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169), 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341)
I20260808 15:06:29.074685 27290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.068972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:34597 (local address 127.2.180.195:46169)
0808 15:06:29.069136 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:29.069142 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:29.069656 (+ 514us) server_negotiation.cc:407] Connection header received
0808 15:06:29.069832 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:29.069835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:29.069891 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:29.069960 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:29.070504 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.071203 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.073019 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.073788 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.073932 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:29.074004 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:29.074315 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 15:06:29.074435 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 15:06:29.074738 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.068930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:34225 (local address 127.2.180.194:41341)
0808 15:06:29.069125 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:29.069131 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:29.069819 (+ 688us) server_negotiation.cc:407] Connection header received
0808 15:06:29.069974 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:29.069979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:29.070038 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:29.070118 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:06:29.070692 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.071371 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.073115 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.073869 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.074009 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:29.074092 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:29.074368 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 15:06:29.074489 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:06:29.074909 27310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.068731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:46169 (local address 127.2.180.193:34597)
0808 15:06:29.069545 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.069565 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:29.069710 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:29.069993 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:29.069997 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:29.070022 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:29.070368 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:29.070382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.071340 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.071344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:29.072872 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:29.072888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.073989 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.073995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:29.074161 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:29.074188 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 15:06:29.074268 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 15:06:29.074391 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":20,"thread_start_us":475,"threads_started":1}
I20260808 15:06:29.075136 27315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.068988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41341 (local address 127.2.180.193:34225)
0808 15:06:29.069715 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.069729 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:29.069846 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:29.070157 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:29.070162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:29.070189 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:06:29.070540 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:29.070549 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.071517 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.071521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:29.072955 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:29.072967 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.074087 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.074093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:29.074227 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:29.074249 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 15:06:29.074326 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 15:06:29.074442 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":1,"thread_start_us":435,"threads_started":1}
I20260808 15:06:29.075856 27214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3134a55adbc14f28a5ea1803fd1a053a" candidate_uuid: "b5428147c9ba4b15ac2bcb8585f99562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec9fb1844d67422d9882744235896c9e" is_pre_election: true
I20260808 15:06:29.076056 27017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3134a55adbc14f28a5ea1803fd1a053a" candidate_uuid: "b5428147c9ba4b15ac2bcb8585f99562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489d2692462746a1a98ed5ed71881b34" is_pre_election: true
I20260808 15:06:29.076282 27214 raft_consensus.cc:2454] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5428147c9ba4b15ac2bcb8585f99562 in term 0.
I20260808 15:06:29.076519 27017 raft_consensus.cc:2454] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5428147c9ba4b15ac2bcb8585f99562 in term 0.
I20260808 15:06:29.076889 26758 leader_election.cc:302] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5428147c9ba4b15ac2bcb8585f99562, ec9fb1844d67422d9882744235896c9e; no voters:
I20260808 15:06:29.077279 27306 raft_consensus.cc:2781] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:06:29.077423 27306 raft_consensus.cc:482] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:06:29.077521 27306 raft_consensus.cc:3037] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:06:29.079597 27306 raft_consensus.cc:504] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:29.080140 27306 leader_election.cc:288] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 1 election: Requested vote from peers ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169), 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341)
I20260808 15:06:29.080555 27017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3134a55adbc14f28a5ea1803fd1a053a" candidate_uuid: "b5428147c9ba4b15ac2bcb8585f99562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "489d2692462746a1a98ed5ed71881b34"
I20260808 15:06:29.080570 27214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3134a55adbc14f28a5ea1803fd1a053a" candidate_uuid: "b5428147c9ba4b15ac2bcb8585f99562" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec9fb1844d67422d9882744235896c9e"
I20260808 15:06:29.080760 27017 raft_consensus.cc:3037] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:06:29.080758 27214 raft_consensus.cc:3037] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:06:29.083874 27214 raft_consensus.cc:2454] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5428147c9ba4b15ac2bcb8585f99562 in term 1.
I20260808 15:06:29.083869 27017 raft_consensus.cc:2454] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5428147c9ba4b15ac2bcb8585f99562 in term 1.
I20260808 15:06:29.084255 26760 leader_election.cc:302] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 489d2692462746a1a98ed5ed71881b34, b5428147c9ba4b15ac2bcb8585f99562; no voters:
I20260808 15:06:29.084534 27306 raft_consensus.cc:2781] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:06:29.084920 27306 raft_consensus.cc:686] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 LEADER]: Becoming Leader. State: Replica: b5428147c9ba4b15ac2bcb8585f99562, State: Running, Role: LEADER
I20260808 15:06:29.085383 27306 consensus_queue.cc:237] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:29.089600 26648 catalog_manager.cc:5495] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5428147c9ba4b15ac2bcb8585f99562 (127.2.180.193). New cstate: current_term: 1 leader_uuid: "b5428147c9ba4b15ac2bcb8585f99562" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } health_report { overall_health: UNKNOWN } } }
W20260808 15:06:29.108421 27272 tablet.cc:2369] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:06:29.137409 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.132669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38232 (local address 127.2.180.193:34495)
0808 15:06:29.132828 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:29.132832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:29.132902 (+ 70us) server_negotiation.cc:407] Connection header received
0808 15:06:29.133062 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:29.133067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:29.133140 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:29.133257 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:06:29.133529 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.134216 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.135587 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.136008 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.136097 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:29.136520 (+ 423us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:06:29.137005 (+ 485us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:06:29.137114 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:29.137150 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 15:06:29.137207 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:06:29.144824 27017 raft_consensus.cc:1261] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 1 FOLLOWER]: Refusing update from remote peer b5428147c9ba4b15ac2bcb8585f99562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:06:29.144827 27214 raft_consensus.cc:1261] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 1 FOLLOWER]: Refusing update from remote peer b5428147c9ba4b15ac2bcb8585f99562: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:06:29.145728 27317 consensus_queue.cc:1035] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:06:29.146055 27306 consensus_queue.cc:1035] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:06:29.156044 27320 mvcc.cc:205] Tried to move back new op lower bound from 7316281709128646656 to 7316281708897587200. Current Snapshot: MvccSnapshot[applied={T|T < 7316281709128646656}]
I20260808 15:06:29.168633 26649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:06:29.168862 26649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:06:29.172144 26649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Sending DeleteTablet for 3 replicas of tablet 3134a55adbc14f28a5ea1803fd1a053a
I20260808 15:06:29.173147 27184 tablet_service.cc:1514] Processing DeleteTablet for tablet 3134a55adbc14f28a5ea1803fd1a053a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:06:29 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:44280
I20260808 15:06:29.173324 26991 tablet_service.cc:1514] Processing DeleteTablet for tablet 3134a55adbc14f28a5ea1803fd1a053a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:06:29 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:35470
I20260808 15:06:29.173889 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 26703
I20260808 15:06:29.173831 26820 tablet_service.cc:1514] Processing DeleteTablet for tablet 3134a55adbc14f28a5ea1803fd1a053a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:06:29 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:38222
I20260808 15:06:29.193573 27293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.174677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.2.180.193:34495)
0808 15:06:29.175238 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:29.175245 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:29.175261 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:06:29.175316 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:29.175321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:29.175398 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:29.175481 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:29.177602 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.178891 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.180884 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.181557 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.181667 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:29.185207 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:29.185222 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:29.185238 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:29.185287 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:29.189417 (+ 4130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:29.190345 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:29.190354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:29.190359 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:29.190537 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:29.190991 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:29.190996 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:29.191000 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:29.191467 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:29.191821 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:29.192950 (+ 1129us) server_negotiation.cc:299] Negotiation successful
0808 15:06:29.193181 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444}
I20260808 15:06:29.194255 26896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:29.194482 27336 tablet_replica.cc:330] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: stopping tablet replica
I20260808 15:06:29.194921 27336 raft_consensus.cc:2229] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:06:29.195425 27336 raft_consensus.cc:2258] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:06:29.195924 27335 tablet_replica.cc:330] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: stopping tablet replica
I20260808 15:06:29.196317 27335 raft_consensus.cc:2229] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:06:29.196815 27335 raft_consensus.cc:2258] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:06:29.198266 27336 ts_tablet_manager.cc:1905] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:06:29.198585 27335 ts_tablet_manager.cc:1905] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:06:29.203222 27335 ts_tablet_manager.cc:1918] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:06:29.203353 27335 log.cc:1196] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/3134a55adbc14f28a5ea1803fd1a053a
I20260808 15:06:29.203866 27335 ts_tablet_manager.cc:1939] T 3134a55adbc14f28a5ea1803fd1a053a P ec9fb1844d67422d9882744235896c9e: Deleting consensus metadata
I20260808 15:06:29.205107 27336 ts_tablet_manager.cc:1918] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:06:29.205260 27336 log.cc:1196] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/3134a55adbc14f28a5ea1803fd1a053a
I20260808 15:06:29.205341 26630 catalog_manager.cc:4841] TS ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169): tablet 3134a55adbc14f28a5ea1803fd1a053a (table test-table [id=07b5589a59cd4fc6ac61addf1c8adc06]) successfully deleted
I20260808 15:06:29.205742 27336 ts_tablet_manager.cc:1939] T 3134a55adbc14f28a5ea1803fd1a053a P 489d2692462746a1a98ed5ed71881b34: Deleting consensus metadata
I20260808 15:06:29.207247 26631 catalog_manager.cc:4841] TS 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341): tablet 3134a55adbc14f28a5ea1803fd1a053a (table test-table [id=07b5589a59cd4fc6ac61addf1c8adc06]) successfully deleted
I20260808 15:06:29.290838 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 26703
W20260808 15:06:29.313894 26632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.180.193:34495 (error 108)
I20260808 15:06:29.315025 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 26908
I20260808 15:06:29.315519 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.315122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38252)
0808 15:06:29.315288 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.315386 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 15:06:29.315922 26632 catalog_manager.cc:4584] TS b5428147c9ba4b15ac2bcb8585f99562 (127.2.180.193:34495): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3134a55adbc14f28a5ea1803fd1a053a: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
I20260808 15:06:29.327402 27294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.315847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35474 (local address 127.2.180.194:41341)
0808 15:06:29.316026 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:29.316032 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:29.316052 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:06:29.316119 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:29.316124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:29.316206 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:29.316303 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:29.317552 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.318253 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.319504 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.320101 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.320195 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:29.322569 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:29.322579 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:29.322592 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:29.322616 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:29.325207 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:29.325850 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:29.325856 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:29.325858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:29.325971 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:29.326239 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:29.326242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:29.326244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:29.326558 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:29.326793 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:29.326935 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 15:06:29.327081 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:06:29.328068 27076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:29.369349 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.368866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38262)
0808 15:06:29.369073 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.369170 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 15:06:29.370288 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.369920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38272)
0808 15:06:29.370068 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.370164 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 15:06:29.413843 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 26908
I20260808 15:06:29.430624 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.430099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38280)
0808 15:06:29.430324 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.430437 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 15:06:29.431632 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.431238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38282)
0808 15:06:29.431393 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.431473 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 15:06:29.438854 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 27086
I20260808 15:06:29.451191 27290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.439568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44284 (local address 127.2.180.195:46169)
0808 15:06:29.439750 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:29.439755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:29.439772 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:06:29.439884 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:29.439888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:29.439972 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:29.440055 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:29.441345 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.442041 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.443308 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:29.443704 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:29.443790 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:29.446284 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:29.446295 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:29.446313 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:29.446338 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:29.448822 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:29.449430 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:29.449435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:29.449438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:29.449551 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:29.449838 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:29.449840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:29.449842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:29.450175 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:29.450386 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:29.450637 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 15:06:29.450857 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:06:29.451861 27264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:29.512756 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.512312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38290)
0808 15:06:29.512510 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.512618 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 15:06:29.513821 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.513440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38306)
0808 15:06:29.513631 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.513717 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 15:06:29.552676 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 27086
I20260808 15:06:29.574815 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 26605
I20260808 15:06:29.575958 26691 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:06:29.687521 27291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:29.687036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:34495 (local address 127.0.0.1:38320)
0808 15:06:29.687204 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:29.687319 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:34495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 15:06:29.687505 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 26605
I20260808 15:06:49.712069 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46351
--webserver_interface=127.2.180.254
--webserver_port=38001
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:46351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:06:50.041970 27347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:50.042230 27347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:50.042322 27347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:50.049116 27347 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 15:06:50.049194 27347 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 15:06:50.049237 27347 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 15:06:50.049288 27347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:06:50.049340 27347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:50.049403 27347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:50.049446 27347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:06:50.049502 27347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:06:50.058532 27347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:46351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=38001
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:50.059557 27347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:50.060708 27347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:50.071346 27354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:50.071388 27356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:50.071439 27359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:50.072212 27347 server_base.cc:1034] running on GCE node
I20260808 15:06:50.072881 27347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:50.074077 27347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:50.075290 27347 hybrid_clock.cc:648] HybridClock initialized: now 1786201610075227 us; error 39 us; skew 500 ppm
I20260808 15:06:50.075582 27347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:50 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201610, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:50.081353 27347 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:06:50.082664 27347 webserver.cc:460] Webserver started at http://127.2.180.254:38001/ using document root <none> and password file <none>
I20260808 15:06:50.083276 27347 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:50.083377 27347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:50.088547 27347 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:06:50.091166 27367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:50.092394 27347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:06:50.092549 27347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "7dddce20179543428aa99988cee5a142"
format_stamp: "Formatted at 2026-08-08 15:06:04 on dist-test-slave-jlzn"
server_key: "c580af29432c8092b8fe7eedf0474187"
server_key_iv: "d392c5d9339b85ddad6409d490edfa32"
server_key_version: "encryptionkey@0"
I20260808 15:06:50.093446 27347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:50.113147 27347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:50.116688 27347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:50.116925 27347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:50.133792 27347 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:46351
I20260808 15:06:50.133845 27437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:46351 every 8 connection(s)
I20260808 15:06:50.135133 27347 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:06:50.138618 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 27347
I20260808 15:06:50.139688 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:34495
--local_ip_for_outbound_sockets=127.2.180.193
--tserver_master_addrs=127.2.180.254:46351
--webserver_port=35125
--webserver_interface=127.2.180.193
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:06:50.142949 27438 sys_catalog.cc:263] Verifying existing consensus state
I20260808 15:06:50.147048 27438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Bootstrap starting.
I20260808 15:06:50.167738 27438 log.cc:824] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:50.183676 27438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:06:50.184191 27438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Bootstrap complete.
I20260808 15:06:50.192937 27438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:50.193786 27438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dddce20179543428aa99988cee5a142, State: Initialized, Role: FOLLOWER
I20260808 15:06:50.194380 27438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:50.194613 27438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:06:50.194718 27438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:06:50.194890 27438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 2 FOLLOWER]: Advancing to term 3
I20260808 15:06:50.198760 27438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:50.199201 27438 leader_election.cc:302] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dddce20179543428aa99988cee5a142; no voters:
I20260808 15:06:50.199823 27438 leader_election.cc:288] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 15:06:50.199972 27444 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 15:06:50.201823 27438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:06:50.206956 27444 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [term 3 LEADER]: Becoming Leader. State: Replica: 7dddce20179543428aa99988cee5a142, State: Running, Role: LEADER
I20260808 15:06:50.207522 27444 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } }
I20260808 15:06:50.210074 27447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7dddce20179543428aa99988cee5a142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } } }
I20260808 15:06:50.210454 27447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: This master's current role is: LEADER
I20260808 15:06:50.215178 27468 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:06:50.215204 27449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dddce20179543428aa99988cee5a142. Latest consensus state: current_term: 3 leader_uuid: "7dddce20179543428aa99988cee5a142" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dddce20179543428aa99988cee5a142" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46351 } } }
I20260808 15:06:50.217311 27449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142 [sys.catalog]: This master's current role is: LEADER
I20260808 15:06:50.227974 27468 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:06:50.229429 27468 catalog_manager.cc:1259] Loaded cluster ID: ba4cdf1f85e6405584b737a66a7b2adf
I20260808 15:06:50.229578 27468 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:06:50.233548 27468 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:06:50.235114 27468 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Loaded TSK: 0
I20260808 15:06:50.235329 27468 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7dddce20179543428aa99988cee5a142: Loaded TSK: 1
F20260808 15:06:50.236505 27468 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fa020d9b9ad google::LogMessage::Fail() at ??:0
@ 0x7fa020d9e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fa020d9b4de google::LogMessage::Flush() at ??:0
@ 0x7fa020d9ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fa024aff7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fa024aee73e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fa0367ca1c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fa0367c9783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fa0368552a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fa0368550c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fa03689f5e9 std::function<>::operator()() at ??:0
@ 0x7fa0367cdcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fa0367cbc36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fa036853610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fa036853439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa0239e7345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fa0239ea91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fa0239ea799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fa0239c5eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fa026c9f6db start_thread at ??:0
@ 0x7fa01fc3561f clone at ??:0
W20260808 15:06:50.521623 27442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:50.521885 27442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:50.522009 27442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:50.529121 27442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:50.529309 27442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:50.529518 27442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:06:50.539038 27442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:34495
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=35125
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:50.540256 27442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:50.541497 27442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:50.553705 27519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:50.553795 27515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:50.553758 27516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:50.554481 27442 server_base.cc:1034] running on GCE node
I20260808 15:06:50.555027 27442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:50.555909 27442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:50.557118 27442 hybrid_clock.cc:648] HybridClock initialized: now 1786201610557079 us; error 32 us; skew 500 ppm
I20260808 15:06:50.557410 27442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:50 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201610, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:50.560339 27442 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:06:50.561691 27442 webserver.cc:460] Webserver started at http://127.2.180.193:35125/ using document root <none> and password file <none>
I20260808 15:06:50.562302 27442 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:50.562409 27442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:50.567382 27442 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 15:06:50.570158 27534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:50.571343 27442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 15:06:50.571491 27442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "b5428147c9ba4b15ac2bcb8585f99562"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "1e38634394653c0408a835c144aa2e63"
server_key_iv: "c49c0638b21fc01b51cc77030ad19c53"
server_key_version: "encryptionkey@0"
I20260808 15:06:50.572468 27442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:50.590480 27442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:50.594044 27442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:50.594281 27442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:50.595153 27442 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:50.596860 27544 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 15:06:50.602087 27442 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 15:06:50.602193 27442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 15:06:50.602304 27442 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 15:06:50.605371 27442 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 15:06:50.605448 27442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260808 15:06:50.605788 27544 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Bootstrap starting.
I20260808 15:06:50.632398 27544 log.cc:824] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Log is configured to *not* fsync() on all Append() calls
I20260808 15:06:50.643157 27442 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:34495
I20260808 15:06:50.643196 27688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:34495 every 8 connection(s)
I20260808 15:06:50.644173 27544 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 15:06:50.644377 27442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:06:50.644692 27544 tablet_bootstrap.cc:492] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Bootstrap complete.
I20260808 15:06:50.645818 27544 ts_tablet_manager.cc:1397] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Time spent bootstrapping tablet: real 0.040s user 0.029s sys 0.008s
I20260808 15:06:50.652217 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:50.650186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:58935)
0808 15:06:50.651026 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:50.651277 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":382,"threads_started":1}
W20260808 15:06:50.652776 27689 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:46351 (0 consecutive failures): Network error: Failed to ping master at 127.2.180.254:46351: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
I20260808 15:06:50.653319 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 27442
I20260808 15:06:50.653779 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:50.653354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:58203)
0808 15:06:50.653553 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:50.653644 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260808 15:06:50.653972 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:41341
--local_ip_for_outbound_sockets=127.2.180.194
--tserver_master_addrs=127.2.180.254:46351
--webserver_port=41937
--webserver_interface=127.2.180.194
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:06:50.654764 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:50.654331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:48571)
0808 15:06:50.654490 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:50.654597 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 15:06:50.655016 27689 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 15:06:50.654075 27544 raft_consensus.cc:348] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:50.655279 27544 raft_consensus.cc:729] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5428147c9ba4b15ac2bcb8585f99562, State: Initialized, Role: FOLLOWER
I20260808 15:06:50.655849 27544 consensus_queue.cc:260] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:50.657245 27544 ts_tablet_manager.cc:1428] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562: Time spent starting tablet: real 0.011s user 0.009s sys 0.004s
I20260808 15:06:50.657375 27694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:50.656977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:48291)
0808 15:06:50.657130 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:50.657210 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 15:06:50.658667 27698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:50.657800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:60435)
0808 15:06:50.658290 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:50.658444 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":216,"threads_started":1}
W20260808 15:06:50.976351 27700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:50.976619 27700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:50.976752 27700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:50.983523 27700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:50.983637 27700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:50.983785 27700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:06:50.992951 27700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:41341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=41937
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:50.994055 27700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:50.995234 27700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:51.007534 27714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:51.007573 27716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:51.007604 27713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:51.008347 27700 server_base.cc:1034] running on GCE node
I20260808 15:06:51.008895 27700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:51.009711 27700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:51.010911 27700 hybrid_clock.cc:648] HybridClock initialized: now 1786201611010868 us; error 32 us; skew 500 ppm
I20260808 15:06:51.011204 27700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:51 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201611, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:51.014210 27700 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:06:51.015539 27700 webserver.cc:460] Webserver started at http://127.2.180.194:41937/ using document root <none> and password file <none>
I20260808 15:06:51.016222 27700 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:51.016335 27700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:51.021252 27700 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:06:51.023680 27726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:51.024873 27700 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:06:51.025018 27700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "489d2692462746a1a98ed5ed71881b34"
format_stamp: "Formatted at 2026-08-08 15:06:05 on dist-test-slave-jlzn"
server_key: "bbcb43966076d7702c0cf047a29fcd60"
server_key_iv: "bdf956185cb54637a03bf1b874aca6c6"
server_key_version: "encryptionkey@0"
I20260808 15:06:51.025916 27700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:51.038527 27700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:51.042328 27700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:51.042559 27700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:51.043402 27700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:51.044612 27700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:06:51.044698 27700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:51.044799 27700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:06:51.044875 27700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:51.074329 27700 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:41341
I20260808 15:06:51.074396 27885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:41341 every 8 connection(s)
I20260808 15:06:51.075618 27700 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:06:51.078572 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 27700
I20260808 15:06:51.079322 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:46169
--local_ip_for_outbound_sockets=127.2.180.195
--tserver_master_addrs=127.2.180.254:46351
--webserver_port=43021
--webserver_interface=127.2.180.195
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:06:51.080452 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:51.078308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:47207)
0808 15:06:51.079116 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:51.079369 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":300,"threads_started":1}
W20260808 15:06:51.081127 27886 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:46351 (0 consecutive failures): Network error: Failed to ping master at 127.2.180.254:46351: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
I20260808 15:06:51.081894 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:51.081549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:41007)
0808 15:06:51.081695 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:51.081771 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 15:06:51.082665 27888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:51.082373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:50671)
0808 15:06:51.082485 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:51.082555 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260808 15:06:51.082875 27886 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 15:06:51.400143 27889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:06:51.400389 27889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:06:51.400506 27889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:06:51.407397 27889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:06:51.407490 27889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:06:51.407629 27889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:06:51.416742 27889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:46169
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=43021
--tserver_master_addrs=127.2.180.254:46351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:06:51.417898 27889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:06:51.419097 27889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:06:51.431246 27896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:51.431690 27902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:06:51.431720 27899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:06:51.432334 27889 server_base.cc:1034] running on GCE node
I20260808 15:06:51.432886 27889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:06:51.433751 27889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:06:51.434952 27889 hybrid_clock.cc:648] HybridClock initialized: now 1786201611434915 us; error 23 us; skew 500 ppm
I20260808 15:06:51.435225 27889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:06:51 dist-test-slave-jlzn krb5kdc[25852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201611, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:06:51.438235 27889 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:06:51.439502 27889 webserver.cc:460] Webserver started at http://127.2.180.195:43021/ using document root <none> and password file <none>
I20260808 15:06:51.440178 27889 fs_manager.cc:359] Metadata directory not provided
I20260808 15:06:51.440294 27889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:06:51.445382 27889 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 15:06:51.447921 27911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:51.449095 27889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:06:51.449244 27889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "ec9fb1844d67422d9882744235896c9e"
format_stamp: "Formatted at 2026-08-08 15:06:06 on dist-test-slave-jlzn"
server_key: "924f7b05719df33f59eb7134968d8d2a"
server_key_iv: "16640fb457ffe8401401cc1a095f9107"
server_key_version: "encryptionkey@0"
I20260808 15:06:51.450153 27889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:06:51.473176 27889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:06:51.477313 27889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:06:51.477546 27889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:06:51.478430 27889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:06:51.479686 27889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:06:51.479780 27889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:51.479907 27889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:06:51.479987 27889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:06:51.509423 27889 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:46169
I20260808 15:06:51.509491 28085 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:46169 every 8 connection(s)
I20260808 15:06:51.510792 27889 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:06:51.514446 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 27889
I20260808 15:06:51.515636 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:51.513475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:34417)
0808 15:06:51.514288 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:51.514585 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":265,"threads_started":1}
W20260808 15:06:51.516477 28088 heartbeater.cc:644] Failed to heartbeat to 127.2.180.254:46351 (0 consecutive failures): Network error: Failed to ping master at 127.2.180.254:46351: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
I20260808 15:06:51.517343 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:51.516897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:47143)
0808 15:06:51.517066 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:51.517223 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 15:06:51.518175 28091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:51.517856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:44183)
0808 15:06:51.517992 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:51.518070 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 15:06:51.518396 28088 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 15:06:51.660084 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:51.659375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:33925)
0808 15:06:51.659751 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:51.659857 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":182,"threads_started":1}
I20260808 15:06:52.084093 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:52.083320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:55141)
0808 15:06:52.083747 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:52.083873 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":234,"threads_started":1}
I20260808 15:06:52.524703 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:52.519035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:36431)
0808 15:06:52.524258 (+ 5223us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:52.524385 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5071,"thread_start_us":4911,"threads_started":1}
I20260808 15:06:52.529510 28099 raft_consensus.cc:482] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:06:52.529824 28099 raft_consensus.cc:504] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:52.531911 28099 leader_election.cc:288] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169), 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341)
Aug 08 15:06:52 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201610, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
Aug 08 15:06:52 dist-test-slave-jlzn krb5kdc[25852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201610, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:06:52.556599 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:52.532116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:46169 (local address 127.2.180.193:38661)
0808 15:06:52.532662 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:52.532710 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:52.533279 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:52.535767 (+ 2488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:52.535778 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:52.536966 (+ 1188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:06:52.537877 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:52.537905 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:52.540310 (+ 2405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:52.540316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:52.542246 (+ 1930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:52.542262 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:52.543557 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:52.543564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:52.543870 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:52.544621 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:06:52.544653 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:06:52.547474 (+ 2821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:06:52.552267 (+ 4793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:52.552275 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:52.552288 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:52.552910 (+ 622us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:52.553570 (+ 660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:52.553574 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:52.553577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:52.553757 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:52.554971 (+ 1214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:06:52.554980 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:06:52.555353 (+ 373us) client_negotiation.cc:769] Sending connection context
0808 15:06:52.555642 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 15:06:52.556129 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"spinlock_wait_cycles":36736,"thread_start_us":1276,"threads_started":1}
I20260808 15:06:52.558022 28104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:52.532660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:38661 (local address 127.2.180.195:46169)
0808 15:06:52.533216 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:52.533230 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:52.533278 (+ 48us) server_negotiation.cc:407] Connection header received
0808 15:06:52.535011 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:52.535019 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:52.535333 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:52.535608 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:52.538108 (+ 2500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:52.540061 (+ 1953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:52.542423 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:52.543345 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:52.543517 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:52.547784 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:52.547795 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:52.547839 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:52.547884 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:52.552087 (+ 4203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:52.553090 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:52.553099 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:52.553107 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:52.553429 (+ 322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:52.553856 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:52.553862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:52.553865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:52.554719 (+ 854us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:52.554896 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:52.557249 (+ 2353us) server_negotiation.cc:299] Negotiation successful
0808 15:06:52.557642 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":596,"threads_started":1}
I20260808 15:06:52.558606 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:52.532214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41341 (local address 127.2.180.193:34539)
0808 15:06:52.532777 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:52.532794 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:06:52.533283 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:06:52.545105 (+ 11822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:06:52.545110 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:06:52.545711 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:06:52.546034 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:52.546042 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:52.547853 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:52.547858 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:52.548862 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:06:52.548874 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:06:52.549821 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:06:52.549828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:06:52.550006 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:52.550500 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:06:52.550517 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:06:52.552541 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:06:52.556034 (+ 3493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:52.556039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:52.556043 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:52.556413 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:52.556865 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:06:52.556870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:06:52.556873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:06:52.557054 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:06:52.557743 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:06:52.557748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:06:52.557886 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 15:06:52.557996 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 15:06:52.558240 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":370,"threads_started":1}
I20260808 15:06:52.559351 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:52.533042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:34539 (local address 127.2.180.194:41341)
0808 15:06:52.542567 (+ 9525us) server_negotiation.cc:206] Beginning negotiation
0808 15:06:52.542580 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:06:52.542617 (+ 37us) server_negotiation.cc:407] Connection header received
0808 15:06:52.544552 (+ 1935us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:06:52.544561 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:06:52.544868 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:06:52.545120 (+ 252us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:06:52.546193 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:52.547697 (+ 1504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:52.548988 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:06:52.549559 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:06:52.549793 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:06:52.552770 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:06:52.552777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:06:52.552788 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:06:52.552821 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:06:52.555917 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:52.556541 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:52.556547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:52.556552 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:52.556730 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:06:52.557168 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:06:52.557173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:06:52.557177 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:06:52.557615 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:06:52.557790 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 15:06:52.558506 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 15:06:52.559085 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9336,"thread_start_us":9191,"threads_started":1}
W20260808 15:06:52.561038 27535 leader_election.cc:341] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169): Not found: Tablet not found: 3134a55adbc14f28a5ea1803fd1a053a
W20260808 15:06:52.561869 27536 leader_election.cc:341] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341): Not found: Tablet not found: 3134a55adbc14f28a5ea1803fd1a053a
I20260808 15:06:52.562001 27536 leader_election.cc:302] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5428147c9ba4b15ac2bcb8585f99562; no voters: 489d2692462746a1a98ed5ed71881b34, ec9fb1844d67422d9882744235896c9e
I20260808 15:06:52.562391 28099 raft_consensus.cc:2726] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:06:52.661350 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:52.660920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:34059)
0808 15:06:52.661101 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:52.661212 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 15:06:53.085587 28106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:53.084894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:41687)
0808 15:06:53.085338 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:53.085424 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":233,"threads_started":1}
I20260808 15:06:53.526186 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:53.525542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:59463)
0808 15:06:53.525946 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:53.526024 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":217,"threads_started":1}
I20260808 15:06:53.662899 28110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:53.662091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:52441)
0808 15:06:53.662636 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:53.662729 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":212,"threads_started":1}
I20260808 15:06:54.086934 28111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:54.086318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:45725)
0808 15:06:54.086697 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:54.086775 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":194,"threads_started":1}
I20260808 15:06:54.402974 28113 raft_consensus.cc:482] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:06:54.403151 28113 raft_consensus.cc:504] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:54.403946 28113 leader_election.cc:288] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169), 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341)
W20260808 15:06:54.404904 27536 leader_election.cc:341] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341): Not found: Tablet not found: 3134a55adbc14f28a5ea1803fd1a053a
W20260808 15:06:54.405058 27535 leader_election.cc:341] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169): Not found: Tablet not found: 3134a55adbc14f28a5ea1803fd1a053a
I20260808 15:06:54.405179 27535 leader_election.cc:302] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5428147c9ba4b15ac2bcb8585f99562; no voters: 489d2692462746a1a98ed5ed71881b34, ec9fb1844d67422d9882744235896c9e
I20260808 15:06:54.405398 28113 raft_consensus.cc:2726] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:06:54.527575 28114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:54.526907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:45085)
0808 15:06:54.527333 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:54.527413 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":227,"threads_started":1}
I20260808 15:06:54.664434 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:54.663670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:55037)
0808 15:06:54.664147 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:54.664250 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":240,"threads_started":1}
I20260808 15:06:55.088330 28117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:55.087663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:45471)
0808 15:06:55.088077 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:55.088158 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":210,"threads_started":1}
I20260808 15:06:55.529070 28118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:55.528367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:53467)
0808 15:06:55.528806 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:55.528885 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":212,"threads_started":1}
I20260808 15:06:55.666054 28120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:55.665395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:37225)
0808 15:06:55.665798 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:55.665884 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":208,"threads_started":1}
I20260808 15:06:56.089740 28122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:56.089117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:44779)
0808 15:06:56.089509 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:56.089584 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":200,"threads_started":1}
I20260808 15:06:56.530607 28123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:56.529935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:58387)
0808 15:06:56.530363 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:56.530444 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":227,"threads_started":1}
I20260808 15:06:56.667578 28124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:56.666839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:52257)
0808 15:06:56.667276 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:56.667376 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":258,"threads_started":1}
I20260808 15:06:57.091154 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:57.090462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:53085)
0808 15:06:57.090901 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:57.090980 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":225,"threads_started":1}
I20260808 15:06:57.532241 28130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:57.531453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:57617)
0808 15:06:57.531969 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:57.532052 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":273,"threads_started":1}
I20260808 15:06:57.666411 28131 raft_consensus.cc:482] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:06:57.666628 28131 raft_consensus.cc:504] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:06:57.667223 28131 leader_election.cc:288] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169), 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341)
W20260808 15:06:57.668150 27536 leader_election.cc:341] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341): Not found: Tablet not found: 3134a55adbc14f28a5ea1803fd1a053a
W20260808 15:06:57.668316 27535 leader_election.cc:341] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169): Not found: Tablet not found: 3134a55adbc14f28a5ea1803fd1a053a
I20260808 15:06:57.668463 27535 leader_election.cc:302] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5428147c9ba4b15ac2bcb8585f99562; no voters: 489d2692462746a1a98ed5ed71881b34, ec9fb1844d67422d9882744235896c9e
I20260808 15:06:57.668692 28131 raft_consensus.cc:2726] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:06:57.669576 28132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:57.668958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:40719)
0808 15:06:57.669321 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:57.669406 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":174,"threads_started":1}
I20260808 15:06:58.092741 28134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:58.091963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:56199)
0808 15:06:58.092485 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:58.092568 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":214,"threads_started":1}
I20260808 15:06:58.533840 28135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:58.533037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:57213)
0808 15:06:58.533561 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:58.533655 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":224,"threads_started":1}
I20260808 15:06:58.670955 28139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:58.670320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:53009)
0808 15:06:58.670721 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:58.670805 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":205,"threads_started":1}
I20260808 15:06:59.094179 28146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:59.093505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:58635)
0808 15:06:59.093918 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:59.094006 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":212,"threads_started":1}
I20260808 15:06:59.535404 28147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:59.534653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:40605)
0808 15:06:59.535141 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:59.535234 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":246,"threads_started":1}
I20260808 15:06:59.672433 28150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:06:59.671755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:57723)
0808 15:06:59.672183 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0808 15:06:59.672261 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":239,"threads_started":1}
I20260808 15:07:00.095597 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:00.094930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:44909)
0808 15:07:00.095368 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:00.095449 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":217,"threads_started":1}
I20260808 15:07:00.399523 28153 raft_consensus.cc:482] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:07:00.399709 28153 raft_consensus.cc:504] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ec9fb1844d67422d9882744235896c9e" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 46169 } } peers { permanent_uuid: "b5428147c9ba4b15ac2bcb8585f99562" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 34495 } } peers { permanent_uuid: "489d2692462746a1a98ed5ed71881b34" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41341 } }
I20260808 15:07:00.400314 28153 leader_election.cc:288] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169), 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341)
W20260808 15:07:00.401167 27535 leader_election.cc:341] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ec9fb1844d67422d9882744235896c9e (127.2.180.195:46169): Not found: Tablet not found: 3134a55adbc14f28a5ea1803fd1a053a
W20260808 15:07:00.401364 27536 leader_election.cc:341] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 489d2692462746a1a98ed5ed71881b34 (127.2.180.194:41341): Not found: Tablet not found: 3134a55adbc14f28a5ea1803fd1a053a
I20260808 15:07:00.401477 27536 leader_election.cc:302] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b5428147c9ba4b15ac2bcb8585f99562; no voters: 489d2692462746a1a98ed5ed71881b34, ec9fb1844d67422d9882744235896c9e
I20260808 15:07:00.401722 28153 raft_consensus.cc:2726] T 3134a55adbc14f28a5ea1803fd1a053a P b5428147c9ba4b15ac2bcb8585f99562 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 15:07:00.536993 28154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:00.536270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:60251)
0808 15:07:00.536748 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:00.536838 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":209,"threads_started":1}
I20260808 15:07:00.674152 28156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:00.673385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:56895)
0808 15:07:00.673809 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:00.673927 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":193,"threads_started":1}
I20260808 15:07:01.097023 28158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:01.096395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:54701)
0808 15:07:01.096784 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:01.096859 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":204,"threads_started":1}
I20260808 15:07:01.538444 28160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:01.537764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:53263)
0808 15:07:01.538186 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:01.538268 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":205,"threads_started":1}
I20260808 15:07:01.675599 28162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:01.674921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:39017)
0808 15:07:01.675321 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:01.675400 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":231,"threads_started":1}
I20260808 15:07:02.098457 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:02.097781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:40605)
0808 15:07:02.098222 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:02.098298 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":213,"threads_started":1}
I20260808 15:07:02.539938 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:02.539224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:58957)
0808 15:07:02.539661 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:02.539738 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":205,"threads_started":1}
I20260808 15:07:02.677022 28166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:02.676340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:45147)
0808 15:07:02.676737 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:02.676834 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":207,"threads_started":1}
I20260808 15:07:03.099931 28168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:03.099212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:60903)
0808 15:07:03.099638 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:03.099728 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":208,"threads_started":1}
I20260808 15:07:03.541414 28169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:03.540715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:57273)
0808 15:07:03.541167 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:03.541247 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":239,"threads_started":1}
I20260808 15:07:03.678473 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:03.677782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:51455)
0808 15:07:03.678197 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:03.678277 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":205,"threads_started":1}
I20260808 15:07:04.101361 28173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:04.100685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:39547)
0808 15:07:04.101107 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:04.101203 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":202,"threads_started":1}
I20260808 15:07:04.542954 28175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:04.542197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:35515)
0808 15:07:04.542670 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:04.542782 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":229,"threads_started":1}
I20260808 15:07:04.679916 28178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:04.679216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:39129)
0808 15:07:04.679621 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:04.679701 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":208,"threads_started":1}
I20260808 15:07:05.102772 28180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:05.102080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:44767)
0808 15:07:05.102496 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:05.102586 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":201,"threads_started":1}
I20260808 15:07:05.544497 28181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:05.543757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:40433)
0808 15:07:05.544220 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:05.544311 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":214,"threads_started":1}
I20260808 15:07:05.681579 28182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:05.680894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.193:57641)
0808 15:07:05.681325 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:05.681413 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":217,"threads_started":1}
I20260808 15:07:06.104449 28183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:06.103742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.194:34979)
0808 15:07:06.104206 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:06.104291 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":226,"threads_started":1}
I20260808 15:07:06.525929 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 27442
I20260808 15:07:06.540781 28185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:06.527078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.2.180.193:34495)
0808 15:07:06.527716 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:06.527726 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:06.527760 (+ 34us) server_negotiation.cc:407] Connection header received
0808 15:07:06.527871 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:06.527880 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:06.528179 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:06.528341 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:06.529832 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:06.530885 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:06.532174 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:06.532738 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:06.532854 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:06.535303 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:06.535314 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:06.535327 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:06.535364 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:06.538016 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:06.538695 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:06.538702 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:06.538704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:06.538848 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:06.539218 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:06.539222 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:06.539224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:06.539577 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:06.539904 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:06.540242 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 15:07:06.540493 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":298,"threads_started":1}
I20260808 15:07:06.541956 27684 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:06.546242 28187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:06.545590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46351 (local address 127.2.180.195:36721)
0808 15:07:06.546009 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:06.546107 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.254:46351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":199,"threads_started":1}
I20260808 15:07:06.628201 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 27442
I20260808 15:07:06.654763 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 27700
I20260808 15:07:06.667034 28188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:06.655487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43964 (local address 127.2.180.194:41341)
0808 15:07:06.656145 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:06.656156 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:06.656172 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:07:06.656229 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:06.656233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:06.656294 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:06.656381 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:06.657752 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:06.658516 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:06.659616 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:06.660090 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:06.660187 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:06.662512 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:06.662516 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:06.662520 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:06.662551 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:06.664915 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:06.665504 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:06.665508 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:06.665510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:06.665641 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:06.665921 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:06.665923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:06.665925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:06.666171 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:06.666366 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:06.666545 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 15:07:06.666694 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":437,"threads_started":1}
I20260808 15:07:06.667666 27879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:06.749105 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 27700
I20260808 15:07:06.773908 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 27889
I20260808 15:07:06.787568 28191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:06.774768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45108 (local address 127.2.180.195:46169)
0808 15:07:06.775317 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:06.775322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:06.775339 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:07:06.775396 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:06.775399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:06.775465 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:06.775563 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:06.777212 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:06.778024 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:06.779322 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:06.779759 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:06.780017 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:06.782556 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:06.782562 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:06.782565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:06.782596 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:06.785031 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:06.785652 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:06.785658 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:06.785661 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:06.785803 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:06.786171 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:06.786174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:06.786176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:06.786427 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:06.786698 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:06.786994 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 15:07:06.787200 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":274,"threads_started":1}
I20260808 15:07:06.788259 28077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:06.872392 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 27889
2026-08-08T15:07:06Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64075 ms)
[----------] 30 tests from SecurityITest (288701 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:06 dist-test-slave-jlzn krb5kdc[28198](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:06 dist-test-slave-jlzn krb5kdc[28198](info): set up 2 sockets
Aug 08 15:07:06 dist-test-slave-jlzn krb5kdc[28198](info): commencing operation
krb5kdc: starting...
W20260808 15:07:08.062305 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:08 dist-test-slave-jlzn krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:07:08.297423 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:32907
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:32907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:08.622215 28221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:08.622467 28221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:08.622570 28221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:08.629483 28221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:08.629585 28221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:08.629638 28221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:08.629695 28221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:08.629738 28221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:08.639014 28221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:32907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:32907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:08.640139 28221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:08.641327 28221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:08.652422 28228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:08.652544 28232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:08.652686 28229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:08.653247 28221 server_base.cc:1034] running on GCE node
I20260808 15:07:08.653980 28221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:08.655198 28221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:08.656385 28221 hybrid_clock.cc:648] HybridClock initialized: now 1786201628656337 us; error 37 us; skew 500 ppm
I20260808 15:07:08.656713 28221 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:08 dist-test-slave-jlzn krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201628, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:08.662412 28221 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:07:08.663750 28221 webserver.cc:460] Webserver started at http://127.2.180.254:33467/ using document root <none> and password file <none>
I20260808 15:07:08.664357 28221 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:08.664459 28221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:08.664714 28221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:08.667384 28221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "d8e87836b15c45df8ee275cc412fc2ab"
format_stamp: "Formatted at 2026-08-08 15:07:08 on dist-test-slave-jlzn"
server_key: "99568122bf6f95919bf96e9b6f03e627"
server_key_iv: "b5ce1798a7864639bf66e604c2bb2acd"
server_key_version: "encryptionkey@0"
I20260808 15:07:08.668099 28221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "d8e87836b15c45df8ee275cc412fc2ab"
format_stamp: "Formatted at 2026-08-08 15:07:08 on dist-test-slave-jlzn"
server_key: "99568122bf6f95919bf96e9b6f03e627"
server_key_iv: "b5ce1798a7864639bf66e604c2bb2acd"
server_key_version: "encryptionkey@0"
I20260808 15:07:08.672750 28221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 15:07:08.675956 28239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:08.677124 28221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:07:08.677271 28221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "d8e87836b15c45df8ee275cc412fc2ab"
format_stamp: "Formatted at 2026-08-08 15:07:08 on dist-test-slave-jlzn"
server_key: "99568122bf6f95919bf96e9b6f03e627"
server_key_iv: "b5ce1798a7864639bf66e604c2bb2acd"
server_key_version: "encryptionkey@0"
I20260808 15:07:08.677433 28221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:08.693285 28221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:08.696946 28221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:08.697206 28221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:08.714322 28221 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:32907
I20260808 15:07:08.714339 28312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:32907 every 8 connection(s)
I20260808 15:07:08.715543 28221 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:08.717670 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 28221
I20260808 15:07:08.717883 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:08.718484 2771 external_mini_cluster.cc:1428] Setting key b37cab089545bfbbb1d344b14529cc0d
I20260808 15:07:08.718976 28313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:08.732332 28313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab: Bootstrap starting.
Aug 08 15:07:08 dist-test-slave-jlzn krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:08.735520 28313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:08.736789 28313 log.cc:824] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:08.739698 28313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab: No bootstrap required, opened a new log
I20260808 15:07:08.741923 28317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:08.720235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46850 (local address 127.2.180.254:32907)
0808 15:07:08.721052 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:08.721066 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:08.721113 (+ 47us) server_negotiation.cc:407] Connection header received
0808 15:07:08.723470 (+ 2357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:08.723497 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:08.723984 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:08.724387 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:08.725729 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:08.727609 (+ 1880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:08.729011 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:08.730023 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:08.730213 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:08.734084 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:08.734094 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:08.734111 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:08.734158 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:08.737767 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:08.738450 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:08.738458 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:08.738465 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:08.738681 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:08.739012 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:08.739017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:08.739020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:08.739647 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:08.739989 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:08.740316 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 15:07:08.740720 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":360,"threads_started":1}
I20260808 15:07:08.747925 28313 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e87836b15c45df8ee275cc412fc2ab" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32907 } }
I20260808 15:07:08.748312 28313 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:08.748413 28313 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e87836b15c45df8ee275cc412fc2ab, State: Initialized, Role: FOLLOWER
I20260808 15:07:08.748955 28313 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e87836b15c45df8ee275cc412fc2ab" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32907 } }
I20260808 15:07:08.749186 28313 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:08.749306 28313 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:08.749447 28313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:08.751362 28313 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e87836b15c45df8ee275cc412fc2ab" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32907 } }
I20260808 15:07:08.751839 28313 leader_election.cc:302] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8e87836b15c45df8ee275cc412fc2ab; no voters:
I20260808 15:07:08.752368 28313 leader_election.cc:288] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:08.752471 28319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:08.753098 28319 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [term 1 LEADER]: Becoming Leader. State: Replica: d8e87836b15c45df8ee275cc412fc2ab, State: Running, Role: LEADER
I20260808 15:07:08.753610 28319 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e87836b15c45df8ee275cc412fc2ab" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32907 } }
I20260808 15:07:08.754138 28313 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:08.756141 28323 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8e87836b15c45df8ee275cc412fc2ab. Latest consensus state: current_term: 1 leader_uuid: "d8e87836b15c45df8ee275cc412fc2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e87836b15c45df8ee275cc412fc2ab" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32907 } } }
I20260808 15:07:08.756547 28323 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:08.756999 28320 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8e87836b15c45df8ee275cc412fc2ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8e87836b15c45df8ee275cc412fc2ab" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 32907 } } }
I20260808 15:07:08.757304 28320 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:08.758392 28329 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:08.763988 28329 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:08.772818 28329 catalog_manager.cc:1347] Generated new cluster ID: 65820f48539842d08b6856d1e05ad347
I20260808 15:07:08.772925 28329 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:08.797307 28329 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:08.798455 28329 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:08.806728 28329 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab: Generated new TSK 0
I20260808 15:07:08.807394 28329 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:07:08.876133 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32907
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:09.239327 28351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:09.239598 28351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:09.239717 28351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:09.247125 28351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:09.247223 28351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:09.247377 28351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:07:09.257159 28351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:09.258354 28351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:09.259661 28351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:09.272054 28357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:09.272017 28359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:09.272106 28356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:09.272850 28351 server_base.cc:1034] running on GCE node
I20260808 15:07:09.273463 28351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:09.274315 28351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:09.275516 28351 hybrid_clock.cc:648] HybridClock initialized: now 1786201629275453 us; error 55 us; skew 500 ppm
I20260808 15:07:09.275831 28351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:09 dist-test-slave-jlzn krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201629, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:09.278873 28351 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:07:09.280241 28351 webserver.cc:460] Webserver started at http://127.2.180.193:42329/ using document root <none> and password file <none>
I20260808 15:07:09.280885 28351 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:09.281009 28351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:09.281302 28351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:09.284171 28351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "79c6ccdbcb0340308d3a636cf00f71a0"
format_stamp: "Formatted at 2026-08-08 15:07:09 on dist-test-slave-jlzn"
server_key: "d962ec4c20c6576026df370e4bf62693"
server_key_iv: "cc5357410ccf9797ef91dbc6b9052551"
server_key_version: "encryptionkey@0"
I20260808 15:07:09.284927 28351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "79c6ccdbcb0340308d3a636cf00f71a0"
format_stamp: "Formatted at 2026-08-08 15:07:09 on dist-test-slave-jlzn"
server_key: "d962ec4c20c6576026df370e4bf62693"
server_key_iv: "cc5357410ccf9797ef91dbc6b9052551"
server_key_version: "encryptionkey@0"
I20260808 15:07:09.289783 28351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 15:07:09.293087 28370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:09.294344 28351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:07:09.294494 28351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "79c6ccdbcb0340308d3a636cf00f71a0"
format_stamp: "Formatted at 2026-08-08 15:07:09 on dist-test-slave-jlzn"
server_key: "d962ec4c20c6576026df370e4bf62693"
server_key_iv: "cc5357410ccf9797ef91dbc6b9052551"
server_key_version: "encryptionkey@0"
I20260808 15:07:09.294689 28351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:09.314728 28351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:09.318406 28351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:09.318627 28351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:09.319483 28351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:09.320704 28351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:09.320779 28351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:09.320868 28351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:09.320910 28351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:09.348923 28351 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:42667
I20260808 15:07:09.348984 28540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:42667 every 8 connection(s)
I20260808 15:07:09.350139 28351 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:07:09.360229 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 28351
I20260808 15:07:09.360450 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:07:09.361150 2771 external_mini_cluster.cc:1428] Setting key f348c6660aec7d4a0cf51d2461dc0cb9
Aug 08 15:07:09 dist-test-slave-jlzn krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201629, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:09.369634 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:09.352511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:38599 (local address 127.2.180.254:32907)
0808 15:07:09.352955 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:09.352960 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:09.353764 (+ 804us) server_negotiation.cc:407] Connection header received
0808 15:07:09.355652 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:09.355658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:09.355739 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:09.355853 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:09.357267 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:09.357985 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:09.359475 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:09.360060 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:09.360197 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:09.363826 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:09.363833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:09.363838 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:09.363876 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:09.366781 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:09.367446 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:09.367451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:09.367454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:09.367607 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:09.368023 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:09.368027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:09.368029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:09.368351 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:09.368481 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:09.369108 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 15:07:09.369314 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":228,"threads_started":1}
I20260808 15:07:09.370344 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:09.352920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:32907 (local address 127.2.180.193:38599)
0808 15:07:09.353602 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:09.353642 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:09.355376 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:09.356091 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:09.356099 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:09.356575 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:09.357088 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:09.357105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:09.358136 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:09.358140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:09.359334 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:09.359346 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:09.360298 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:09.360305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:09.360554 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:09.361197 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:09.361230 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:09.363579 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:09.366917 (+ 3338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:09.366923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:09.366933 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:09.367339 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:09.367709 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:09.367712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:09.367715 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:09.367923 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:09.368472 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:09.368477 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:09.368777 (+ 300us) client_negotiation.cc:769] Sending connection context
0808 15:07:09.368990 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 15:07:09.369339 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":253,"threads_started":1}
I20260808 15:07:09.371764 28541 heartbeater.cc:347] Connected to a master server at 127.2.180.254:32907
I20260808 15:07:09.372049 28541 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:09.372704 28541 heartbeater.cc:511] Master 127.2.180.254:32907 requested a full tablet report, sending...
I20260808 15:07:09.374703 28266 ts_manager.cc:194] Registered new tserver with Master: 79c6ccdbcb0340308d3a636cf00f71a0 (127.2.180.193:42667)
I20260808 15:07:09.376649 28266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:38599
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:07:09.419224 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32907
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:09.738428 28554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:09.738696 28554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:09.738830 28554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:09.745723 28554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:09.745819 28554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:09.745963 28554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:07:09.755079 28554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:09.756175 28554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:09.757377 28554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:09.770327 28562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:09.770416 28566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:09.770491 28564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:09.771461 28554 server_base.cc:1034] running on GCE node
I20260808 15:07:09.772145 28554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:09.772972 28554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:09.774186 28554 hybrid_clock.cc:648] HybridClock initialized: now 1786201629774137 us; error 41 us; skew 500 ppm
I20260808 15:07:09.774482 28554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:09 dist-test-slave-jlzn krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201629, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:09.777490 28554 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:07:09.778779 28554 webserver.cc:460] Webserver started at http://127.2.180.194:41379/ using document root <none> and password file <none>
I20260808 15:07:09.779398 28554 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:09.779516 28554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:09.779839 28554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:09.782645 28554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "cfaead5433f04480b8e0233ece35d092"
format_stamp: "Formatted at 2026-08-08 15:07:09 on dist-test-slave-jlzn"
server_key: "6802dd5c669f08fee186322cd00b5ee6"
server_key_iv: "41d6b45777f5fe230ffb344bd7e4b26f"
server_key_version: "encryptionkey@0"
I20260808 15:07:09.783401 28554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "cfaead5433f04480b8e0233ece35d092"
format_stamp: "Formatted at 2026-08-08 15:07:09 on dist-test-slave-jlzn"
server_key: "6802dd5c669f08fee186322cd00b5ee6"
server_key_iv: "41d6b45777f5fe230ffb344bd7e4b26f"
server_key_version: "encryptionkey@0"
I20260808 15:07:09.788519 28554 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 15:07:09.791711 28576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:09.792999 28554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:07:09.793154 28554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "cfaead5433f04480b8e0233ece35d092"
format_stamp: "Formatted at 2026-08-08 15:07:09 on dist-test-slave-jlzn"
server_key: "6802dd5c669f08fee186322cd00b5ee6"
server_key_iv: "41d6b45777f5fe230ffb344bd7e4b26f"
server_key_version: "encryptionkey@0"
I20260808 15:07:09.793329 28554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:09.829227 28554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:09.833364 28554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:09.833629 28554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:09.834555 28554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:09.835856 28554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:09.835942 28554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:09.836078 28554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:09.836138 28554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:09.865876 28554 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:39031
I20260808 15:07:09.865934 28738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:39031 every 8 connection(s)
I20260808 15:07:09.867074 28554 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:07:09.871733 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 28554
I20260808 15:07:09.871927 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:07:09.872321 2771 external_mini_cluster.cc:1428] Setting key 4228f7764cb522d4cbac1806fa2174cc
Aug 08 15:07:09 dist-test-slave-jlzn krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201629, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:09.886175 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:09.869507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:49071 (local address 127.2.180.254:32907)
0808 15:07:09.869712 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:09.869718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:09.870831 (+ 1113us) server_negotiation.cc:407] Connection header received
0808 15:07:09.872727 (+ 1896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:09.872740 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:09.872827 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:09.872917 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:09.874335 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:09.875042 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:09.876580 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:09.877017 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:09.877130 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:09.880415 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:09.880420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:09.880423 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:09.880455 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:09.882944 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:09.883882 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:09.883888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:09.883908 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:09.884071 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:09.884534 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:09.884537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:09.884539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:09.884791 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:09.884909 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:09.885614 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 15:07:09.885841 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:07:09.886787 28745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:09.870013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:32907 (local address 127.2.180.194:49071)
0808 15:07:09.870669 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:09.870704 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:09.872471 (+ 1767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:09.873164 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:09.873172 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:09.873644 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:09.874171 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:09.874189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:09.875194 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:09.875198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:09.876434 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:09.876448 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:09.877235 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:09.877243 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:09.877442 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:09.878017 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:09.878042 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:09.880222 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:09.883122 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:09.883131 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:09.883144 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:09.883725 (+ 581us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:09.884197 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:09.884202 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:09.884205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:09.884435 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:09.884906 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:09.884913 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:09.885247 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 15:07:09.885453 (+ 206us) client_negotiation.cc:240] Negotiation successful
0808 15:07:09.885796 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":261,"threads_started":1}
I20260808 15:07:09.888060 28739 heartbeater.cc:347] Connected to a master server at 127.2.180.254:32907
I20260808 15:07:09.888402 28739 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:09.889176 28739 heartbeater.cc:511] Master 127.2.180.254:32907 requested a full tablet report, sending...
I20260808 15:07:09.890630 28266 ts_manager.cc:194] Registered new tserver with Master: cfaead5433f04480b8e0233ece35d092 (127.2.180.194:39031)
I20260808 15:07:09.891394 28266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:49071
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:07:09.928874 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32907
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:44625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:10.244237 28748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:10.244484 28748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:10.244612 28748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:10.251542 28748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:10.251638 28748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:10.251782 28748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:07:10.261020 28748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:44625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:32907
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:10.262204 28748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:10.263427 28748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:10.275411 28758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:10.275538 28764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:10.275592 28761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:10.276593 28748 server_base.cc:1034] running on GCE node
I20260808 15:07:10.277221 28748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:10.278067 28748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:10.279271 28748 hybrid_clock.cc:648] HybridClock initialized: now 1786201630279219 us; error 48 us; skew 500 ppm
I20260808 15:07:10.279567 28748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:10 dist-test-slave-jlzn krb5kdc[28198](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201630, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:10.282714 28748 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:07:10.284085 28748 webserver.cc:460] Webserver started at http://127.2.180.195:42745/ using document root <none> and password file <none>
I20260808 15:07:10.284680 28748 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:10.284782 28748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:10.285038 28748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:10.287793 28748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "cdee18284e4f4728b1c6a92353716c14"
format_stamp: "Formatted at 2026-08-08 15:07:10 on dist-test-slave-jlzn"
server_key: "941895f1ca9f02c0c71ca30e85bb3cf8"
server_key_iv: "9798f139a9a08c5f05542c61db3d65c7"
server_key_version: "encryptionkey@0"
I20260808 15:07:10.288606 28748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "cdee18284e4f4728b1c6a92353716c14"
format_stamp: "Formatted at 2026-08-08 15:07:10 on dist-test-slave-jlzn"
server_key: "941895f1ca9f02c0c71ca30e85bb3cf8"
server_key_iv: "9798f139a9a08c5f05542c61db3d65c7"
server_key_version: "encryptionkey@0"
I20260808 15:07:10.293447 28748 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 15:07:10.296792 28777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:10.298105 28748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:07:10.298245 28748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "cdee18284e4f4728b1c6a92353716c14"
format_stamp: "Formatted at 2026-08-08 15:07:10 on dist-test-slave-jlzn"
server_key: "941895f1ca9f02c0c71ca30e85bb3cf8"
server_key_iv: "9798f139a9a08c5f05542c61db3d65c7"
server_key_version: "encryptionkey@0"
I20260808 15:07:10.298396 28748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:10.312608 28748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:10.316505 28748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:10.316727 28748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:10.317600 28748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:10.318802 28748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:10.318874 28748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:10.318962 28748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:10.319008 28748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:10.350814 28748 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:44357
I20260808 15:07:10.350849 28946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:44357 every 8 connection(s)
I20260808 15:07:10.352120 28748 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:07:10.358407 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 28748
I20260808 15:07:10.358570 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:07:10.359018 2771 external_mini_cluster.cc:1428] Setting key be32bfdbe0b528eaed368924af9116d2
Aug 08 15:07:10 dist-test-slave-jlzn krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201630, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:10.370798 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.354634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:34989 (local address 127.2.180.254:32907)
0808 15:07:10.354813 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.354817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.355905 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 15:07:10.357679 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.357685 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.357776 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.357864 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:10.359357 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.360392 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.361956 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.362393 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.362503 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.365641 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:10.365648 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:10.365652 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:10.365685 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:10.368030 (+ 2345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:10.368696 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:10.368702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:10.368706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:10.368835 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:10.369232 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:10.369237 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:10.369240 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:10.369502 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:10.369636 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.370253 (+ 617us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.370464 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:07:10.371675 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.355053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:32907 (local address 127.2.180.195:34989)
0808 15:07:10.355743 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:10.355781 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:10.357413 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:10.358098 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:10.358105 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:10.358629 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:10.359193 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.359212 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.360554 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.360567 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.361822 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.361835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.362613 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.362621 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.362839 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.363395 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:10.363417 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:10.365435 (+ 2018us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:10.368169 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:10.368178 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:10.368192 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:10.368584 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:10.368937 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:10.368942 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:10.368946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:10.369125 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:10.369630 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:10.369637 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:10.369921 (+ 284us) client_negotiation.cc:769] Sending connection context
0808 15:07:10.370156 (+ 235us) client_negotiation.cc:240] Negotiation successful
0808 15:07:10.370623 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":250,"threads_started":1}
I20260808 15:07:10.372977 28947 heartbeater.cc:347] Connected to a master server at 127.2.180.254:32907
I20260808 15:07:10.373245 28947 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:10.373904 28947 heartbeater.cc:511] Master 127.2.180.254:32907 requested a full tablet report, sending...
I20260808 15:07:10.375226 28265 ts_manager.cc:194] Registered new tserver with Master: cdee18284e4f4728b1c6a92353716c14 (127.2.180.195:44357)
I20260808 15:07:10.376091 28265 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:34989
I20260808 15:07:10.379329 28541 heartbeater.cc:503] Master 127.2.180.254:32907 was elected leader, sending a full tablet report...
I20260808 15:07:10.385444 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:07:10.401042 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.389586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46860 (local address 127.2.180.254:32907)
0808 15:07:10.389768 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.389774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.390113 (+ 339us) server_negotiation.cc:407] Connection header received
0808 15:07:10.390228 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.390232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.390286 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.390358 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:10.391682 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.392415 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.393620 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.394000 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.394091 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.396439 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:10.396444 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:10.396447 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:10.396474 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:10.398857 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:10.399449 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:10.399453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:10.399455 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:10.399565 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:10.399884 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:10.399887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:10.399889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:10.400175 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:10.400358 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.400594 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.400738 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:07:10.404629 28265 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:07:10.411163 28265 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:07:10.443848 28963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.422765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:39031 (local address 127.0.0.1:39632)
0808 15:07:10.433455 (+ 10690us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:10.433478 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:10.433588 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:10.436947 (+ 3359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:10.436953 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:10.436981 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:10.437414 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.437428 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.439028 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.439033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.441402 (+ 2369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.441417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.442857 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.442864 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.443037 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.443086 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 15:07:10.443227 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 15:07:10.443398 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10549,"mutex_wait_us":94,"thread_start_us":13084,"threads_started":1}
I20260808 15:07:10.444087 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.423472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56566 (local address 127.2.180.195:44357)
0808 15:07:10.434170 (+ 10698us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.434180 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.436389 (+ 2209us) server_negotiation.cc:407] Connection header received
0808 15:07:10.436507 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.436515 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.436778 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.436962 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:10.437905 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.439351 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.441745 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.443000 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.443129 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.443348 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.443554 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.443721 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10525,"thread_start_us":10363,"threads_started":1}
I20260808 15:07:10.444661 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.422919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39632 (local address 127.2.180.194:39031)
0808 15:07:10.436425 (+ 13506us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.436433 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.436459 (+ 26us) server_negotiation.cc:407] Connection header received
0808 15:07:10.436579 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.436587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.436753 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.436926 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:10.437615 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.438911 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.441578 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.442675 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.442809 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.443004 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.444208 (+ 1204us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.444362 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13344,"thread_start_us":13242,"threads_started":1}
I20260808 15:07:10.444020 28962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.422898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:44357 (local address 127.0.0.1:56566)
0808 15:07:10.436142 (+ 13244us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:10.436164 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:10.436279 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:10.437242 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:10.437247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:10.437274 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:10.437718 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.437733 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.439497 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.439503 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.441613 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.441624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.443207 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.443213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.443381 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.443410 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 15:07:10.443495 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 15:07:10.443616 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13119,"mutex_wait_us":3,"thread_start_us":10270,"threads_started":1}
I20260808 15:07:10.444975 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.422567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58862 (local address 127.2.180.193:42667)
0808 15:07:10.437574 (+ 15007us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.437582 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.437603 (+ 21us) server_negotiation.cc:407] Connection header received
0808 15:07:10.437681 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.437686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.437828 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.438027 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:10.438532 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.439960 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.442091 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.443462 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.443587 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.443790 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.444346 (+ 556us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.444673 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14837,"thread_start_us":14686,"threads_started":1}
I20260808 15:07:10.444736 28960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.422283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:42667 (local address 127.0.0.1:58862)
0808 15:07:10.429298 (+ 7015us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:10.429348 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:10.429524 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:10.438018 (+ 8494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:10.438023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:10.438049 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:10.438362 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.438372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.440120 (+ 1748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.440126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.441941 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.441956 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.443974 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.443979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.444142 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.444188 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 15:07:10.444280 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 15:07:10.444395 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6741,"mutex_wait_us":1,"thread_start_us":6601,"threads_started":1}
I20260808 15:07:10.448159 28650 tablet_service.cc:1467] Processing CreateTablet for tablet 9d272a4ce280441aa1e64106d0ac3775 (DEFAULT_TABLE table=test-table [id=abf1d22030d246d4853a6482595cf1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:10.448772 28848 tablet_service.cc:1467] Processing CreateTablet for tablet 9d272a4ce280441aa1e64106d0ac3775 (DEFAULT_TABLE table=test-table [id=abf1d22030d246d4853a6482595cf1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:10.448772 28439 tablet_service.cc:1467] Processing CreateTablet for tablet 9d272a4ce280441aa1e64106d0ac3775 (DEFAULT_TABLE table=test-table [id=abf1d22030d246d4853a6482595cf1f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:10.449335 28650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d272a4ce280441aa1e64106d0ac3775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:10.450132 28848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d272a4ce280441aa1e64106d0ac3775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:10.450151 28439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d272a4ce280441aa1e64106d0ac3775. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:10.460686 28966 tablet_bootstrap.cc:492] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Bootstrap starting.
I20260808 15:07:10.463644 28966 tablet_bootstrap.cc:654] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:10.464100 28968 tablet_bootstrap.cc:492] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Bootstrap starting.
I20260808 15:07:10.465008 28966 log.cc:824] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:10.467666 28968 tablet_bootstrap.cc:654] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:10.467687 28966 tablet_bootstrap.cc:492] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: No bootstrap required, opened a new log
I20260808 15:07:10.467967 28966 ts_tablet_manager.cc:1397] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260808 15:07:10.467967 28969 tablet_bootstrap.cc:492] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Bootstrap starting.
I20260808 15:07:10.468734 28968 log.cc:824] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:10.471244 28969 tablet_bootstrap.cc:654] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:10.471441 28968 tablet_bootstrap.cc:492] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: No bootstrap required, opened a new log
I20260808 15:07:10.471681 28968 ts_tablet_manager.cc:1397] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 15:07:10.472422 28969 log.cc:824] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:10.475178 28969 tablet_bootstrap.cc:492] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: No bootstrap required, opened a new log
I20260808 15:07:10.475425 28969 ts_tablet_manager.cc:1397] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20260808 15:07:10.477624 28966 raft_consensus.cc:348] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.478039 28966 raft_consensus.cc:374] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:10.478152 28966 raft_consensus.cc:729] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfaead5433f04480b8e0233ece35d092, State: Initialized, Role: FOLLOWER
I20260808 15:07:10.478744 28966 consensus_queue.cc:260] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.478994 28968 raft_consensus.cc:348] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.479421 28968 raft_consensus.cc:374] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:10.479573 28968 raft_consensus.cc:729] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79c6ccdbcb0340308d3a636cf00f71a0, State: Initialized, Role: FOLLOWER
I20260808 15:07:10.479904 28739 heartbeater.cc:503] Master 127.2.180.254:32907 was elected leader, sending a full tablet report...
I20260808 15:07:10.480481 28966 ts_tablet_manager.cc:1428] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260808 15:07:10.480373 28968 consensus_queue.cc:260] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.483021 28968 ts_tablet_manager.cc:1428] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260808 15:07:10.485004 28969 raft_consensus.cc:348] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.485409 28969 raft_consensus.cc:374] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:10.485508 28969 raft_consensus.cc:729] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdee18284e4f4728b1c6a92353716c14, State: Initialized, Role: FOLLOWER
I20260808 15:07:10.486018 28969 consensus_queue.cc:260] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.487010 28947 heartbeater.cc:503] Master 127.2.180.254:32907 was elected leader, sending a full tablet report...
I20260808 15:07:10.487499 28969 ts_tablet_manager.cc:1428] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
W20260808 15:07:10.602241 28542 tablet.cc:2369] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:07:10.603670 28948 tablet.cc:2369] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:07:10.619005 28740 tablet.cc:2369] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:07:10.619278 28978 raft_consensus.cc:482] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:07:10.619572 28978 raft_consensus.cc:504] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.621213 28978 leader_election.cc:288] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79c6ccdbcb0340308d3a636cf00f71a0 (127.2.180.193:42667), cdee18284e4f4728b1c6a92353716c14 (127.2.180.195:44357)
I20260808 15:07:10.627596 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.621653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:33537 (local address 127.2.180.195:44357)
0808 15:07:10.621917 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.621924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.622359 (+ 435us) server_negotiation.cc:407] Connection header received
0808 15:07:10.622540 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.622544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.622607 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.622680 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:10.623307 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.624106 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.625913 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.626675 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.626826 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.626902 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.627227 (+ 325us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.627349 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260808 15:07:10.627616 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.621513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:58265 (local address 127.2.180.193:42667)
0808 15:07:10.621735 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.621741 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.622522 (+ 781us) server_negotiation.cc:407] Connection header received
0808 15:07:10.622682 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.622686 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.622745 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.622817 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:10.623425 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.624145 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.625925 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.626703 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.626842 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.626915 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.627265 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.627388 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:07:10.627821 28983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.621552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:42667 (local address 127.2.180.194:58265)
0808 15:07:10.622417 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:10.622434 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:10.622556 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:10.622853 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:10.622857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:10.622895 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:10.623265 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.623275 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.624289 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.624292 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.625778 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.625791 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.626958 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.626966 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.627121 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.627143 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 15:07:10.627223 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 15:07:10.627338 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":54,"thread_start_us":541,"threads_started":1}
I20260808 15:07:10.627874 28982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.621480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:44357 (local address 127.2.180.194:33537)
0808 15:07:10.622246 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:10.622263 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:10.622410 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:10.622709 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:10.622713 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:10.622737 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:10.623145 (+ 408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.623156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.624228 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.624232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.625763 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:10.625776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.626919 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.626925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:10.627081 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.627104 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:07:10.627183 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 15:07:10.627303 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":2,"spinlock_wait_cycles":15616,"thread_start_us":441,"threads_started":1}
I20260808 15:07:10.628906 28875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d272a4ce280441aa1e64106d0ac3775" candidate_uuid: "cfaead5433f04480b8e0233ece35d092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdee18284e4f4728b1c6a92353716c14" is_pre_election: true
I20260808 15:07:10.628906 28473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d272a4ce280441aa1e64106d0ac3775" candidate_uuid: "cfaead5433f04480b8e0233ece35d092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" is_pre_election: true
I20260808 15:07:10.629458 28875 raft_consensus.cc:2454] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfaead5433f04480b8e0233ece35d092 in term 0.
I20260808 15:07:10.629458 28473 raft_consensus.cc:2454] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfaead5433f04480b8e0233ece35d092 in term 0.
I20260808 15:07:10.630170 28577 leader_election.cc:302] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6ccdbcb0340308d3a636cf00f71a0, cfaead5433f04480b8e0233ece35d092; no voters:
I20260808 15:07:10.630553 28978 raft_consensus.cc:2781] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:07:10.630682 28978 raft_consensus.cc:482] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:07:10.630772 28978 raft_consensus.cc:3037] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:10.632957 28978 raft_consensus.cc:504] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.633496 28978 leader_election.cc:288] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [CANDIDATE]: Term 1 election: Requested vote from peers 79c6ccdbcb0340308d3a636cf00f71a0 (127.2.180.193:42667), cdee18284e4f4728b1c6a92353716c14 (127.2.180.195:44357)
I20260808 15:07:10.633890 28473 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d272a4ce280441aa1e64106d0ac3775" candidate_uuid: "cfaead5433f04480b8e0233ece35d092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79c6ccdbcb0340308d3a636cf00f71a0"
I20260808 15:07:10.633956 28875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d272a4ce280441aa1e64106d0ac3775" candidate_uuid: "cfaead5433f04480b8e0233ece35d092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cdee18284e4f4728b1c6a92353716c14"
I20260808 15:07:10.634064 28473 raft_consensus.cc:3037] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:10.634142 28875 raft_consensus.cc:3037] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:10.636257 28473 raft_consensus.cc:2454] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfaead5433f04480b8e0233ece35d092 in term 1.
I20260808 15:07:10.636301 28875 raft_consensus.cc:2454] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cfaead5433f04480b8e0233ece35d092 in term 1.
I20260808 15:07:10.636621 28577 leader_election.cc:302] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79c6ccdbcb0340308d3a636cf00f71a0, cfaead5433f04480b8e0233ece35d092; no voters:
I20260808 15:07:10.636897 28978 raft_consensus.cc:2781] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:10.637290 28978 raft_consensus.cc:686] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 1 LEADER]: Becoming Leader. State: Replica: cfaead5433f04480b8e0233ece35d092, State: Running, Role: LEADER
I20260808 15:07:10.637722 28978 consensus_queue.cc:237] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } }
I20260808 15:07:10.641762 28265 catalog_manager.cc:5495] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 reported cstate change: term changed from 0 to 1, leader changed from <none> to cfaead5433f04480b8e0233ece35d092 (127.2.180.194). New cstate: current_term: 1 leader_uuid: "cfaead5433f04480b8e0233ece35d092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cfaead5433f04480b8e0233ece35d092" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 39031 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 } health_report { overall_health: UNKNOWN } } }
I20260808 15:07:10.670675 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.665915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39640 (local address 127.2.180.194:39031)
0808 15:07:10.666080 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.666084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.666234 (+ 150us) server_negotiation.cc:407] Connection header received
0808 15:07:10.666384 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.666387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.666462 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.666584 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:07:10.666839 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.667487 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.668871 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.669280 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.669372 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.669746 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:07:10.670180 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:07:10.670280 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.670356 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.670415 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:07:10.677779 28875 raft_consensus.cc:1261] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 1 FOLLOWER]: Refusing update from remote peer cfaead5433f04480b8e0233ece35d092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:10.677783 28473 raft_consensus.cc:1261] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 1 FOLLOWER]: Refusing update from remote peer cfaead5433f04480b8e0233ece35d092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:10.678659 28984 consensus_queue.cc:1035] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cdee18284e4f4728b1c6a92353716c14" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 44357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:10.678982 28978 consensus_queue.cc:1035] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79c6ccdbcb0340308d3a636cf00f71a0" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 42667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:10.688961 28987 mvcc.cc:205] Tried to move back new op lower bound from 7316281879247822848 to 7316281879095427072. Current Snapshot: MvccSnapshot[applied={T|T < 7316281879247822848}]
I20260808 15:07:10.690999 28989 mvcc.cc:205] Tried to move back new op lower bound from 7316281879247822848 to 7316281879095427072. Current Snapshot: MvccSnapshot[applied={T|T < 7316281879247822848}]
I20260808 15:07:10.700646 28266 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:10.700861 28266 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:10.704263 28266 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d8e87836b15c45df8ee275cc412fc2ab: Sending DeleteTablet for 3 replicas of tablet 9d272a4ce280441aa1e64106d0ac3775
I20260808 15:07:10.705344 28650 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d272a4ce280441aa1e64106d0ac3775 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:10 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:39632
I20260808 15:07:10.705440 28848 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d272a4ce280441aa1e64106d0ac3775 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:10 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:56566
I20260808 15:07:10.705466 28439 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d272a4ce280441aa1e64106d0ac3775 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:10 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:58862
I20260808 15:07:10.708531 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 28351
I20260808 15:07:10.716058 29009 tablet_replica.cc:330] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: stopping tablet replica
I20260808 15:07:10.716473 29009 raft_consensus.cc:2229] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:10.716926 29009 raft_consensus.cc:2258] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:10.717705 29008 tablet_replica.cc:330] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: stopping tablet replica
I20260808 15:07:10.718077 29008 raft_consensus.cc:2229] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:07:10.718758 29008 raft_consensus.cc:2258] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:10.719451 29003 tablet_replica.cc:330] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: stopping tablet replica
I20260808 15:07:10.719820 29003 raft_consensus.cc:2229] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:10.720285 29003 raft_consensus.cc:2258] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:10.722323 29008 ts_tablet_manager.cc:1905] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:10.728058 29003 ts_tablet_manager.cc:1905] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:10.730733 29009 ts_tablet_manager.cc:1905] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:10.734131 29003 ts_tablet_manager.cc:1918] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:10.734282 29003 log.cc:1196] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/9d272a4ce280441aa1e64106d0ac3775
Aug 08 15:07:10 dist-test-slave-jlzn krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:07:10.734798 29003 ts_tablet_manager.cc:1939] T 9d272a4ce280441aa1e64106d0ac3775 P cdee18284e4f4728b1c6a92353716c14: Deleting consensus metadata
I20260808 15:07:10.736986 28248 catalog_manager.cc:4841] TS cdee18284e4f4728b1c6a92353716c14 (127.2.180.195:44357): tablet 9d272a4ce280441aa1e64106d0ac3775 (table test-table [id=abf1d22030d246d4853a6482595cf1f6]) successfully deleted
I20260808 15:07:10.738574 29009 ts_tablet_manager.cc:1918] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:10.738724 29009 log.cc:1196] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/9d272a4ce280441aa1e64106d0ac3775
I20260808 15:07:10.739142 29009 ts_tablet_manager.cc:1939] T 9d272a4ce280441aa1e64106d0ac3775 P 79c6ccdbcb0340308d3a636cf00f71a0: Deleting consensus metadata
I20260808 15:07:10.739535 29008 ts_tablet_manager.cc:1918] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:10.739719 29008 log.cc:1196] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/9d272a4ce280441aa1e64106d0ac3775
I20260808 15:07:10.740254 29008 ts_tablet_manager.cc:1939] T 9d272a4ce280441aa1e64106d0ac3775 P cfaead5433f04480b8e0233ece35d092: Deleting consensus metadata
I20260808 15:07:10.740799 28240 catalog_manager.cc:4841] TS 79c6ccdbcb0340308d3a636cf00f71a0 (127.2.180.193:42667): tablet 9d272a4ce280441aa1e64106d0ac3775 (table test-table [id=abf1d22030d246d4853a6482595cf1f6]) successfully deleted
I20260808 15:07:10.741679 28245 catalog_manager.cc:4841] TS cfaead5433f04480b8e0233ece35d092 (127.2.180.194:39031): tablet 9d272a4ce280441aa1e64106d0ac3775 (table test-table [id=abf1d22030d246d4853a6482595cf1f6]) successfully deleted
I20260808 15:07:10.743021 28961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.710971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58864 (local address 127.2.180.193:42667)
0808 15:07:10.711146 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.711165 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.723996 (+ 12831us) server_negotiation.cc:407] Connection header received
0808 15:07:10.724387 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.724393 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.724466 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.724530 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:10.726263 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.727370 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.729230 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.729911 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.730040 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.735776 (+ 5736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:10.735789 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:10.735864 (+ 75us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:10.735916 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:10.740161 (+ 4245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:10.740939 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:10.740948 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:10.740951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:10.741104 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:10.741454 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:10.741458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:10.741460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:10.741902 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:10.742196 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.742493 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.742690 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:07:10.743639 28531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:10.860524 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 28351
I20260808 15:07:10.893461 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 28554
Aug 08 15:07:10 dist-test-slave-jlzn krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:07:10.910223 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:10.894242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39650 (local address 127.2.180.194:39031)
0808 15:07:10.894417 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:10.894422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:10.894487 (+ 65us) server_negotiation.cc:407] Connection header received
0808 15:07:10.894660 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:10.894687 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:10.894747 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:10.894825 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:10.896546 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.897410 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.899206 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:10.899633 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:10.899722 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:10.904604 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:10.904614 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:10.904631 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:10.904679 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:10.907538 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:10.908258 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:10.908266 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:10.908268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:10.908408 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:10.908778 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:10.908781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:10.908784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:10.909126 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:10.909453 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:10.909700 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 15:07:10.909906 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 15:07:10.910841 28734 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:11.020406 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 28554
I20260808 15:07:11.044495 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 28748
Aug 08 15:07:11 dist-test-slave-jlzn krb5kdc[28198](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:07:11.058723 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:11.045208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.2.180.195:44357)
0808 15:07:11.045368 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:11.045372 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:11.045461 (+ 89us) server_negotiation.cc:407] Connection header received
0808 15:07:11.045626 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:11.045631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:11.045699 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:11.045793 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:11.047096 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:11.047865 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:11.049064 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:11.049431 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:11.049525 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:11.053760 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:11.053775 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:11.053791 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:11.053827 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:11.056585 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:11.057144 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:11.057151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:11.057153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:11.057295 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:11.057571 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:11.057574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:11.057576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:11.057890 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:11.058094 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:11.058303 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 15:07:11.058451 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:07:11.059343 28942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:11.145169 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 28748
I20260808 15:07:11.167816 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 28221
I20260808 15:07:11.168967 28306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:11.275082 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 28221
2026-08-08T15:07:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4405 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T15:07:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:11Z Disabled control of system clock
I20260808 15:07:11.354279 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35579
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:35579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:07:11.671243 29024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:11.671517 29024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:11.671648 29024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:11.678588 29024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:11.678691 29024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:11.678755 29024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:11.678828 29024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:11.678894 29024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:11.687569 29024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:35579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:11.688732 29024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:11.690373 29024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:11.701403 29032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:11.701387 29033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:11.701418 29038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:11.702033 29024 server_base.cc:1034] running on GCE node
I20260808 15:07:11.702791 29024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:11.704133 29024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:11.705338 29024 hybrid_clock.cc:648] HybridClock initialized: now 1786201631705289 us; error 43 us; skew 500 ppm
I20260808 15:07:11.705675 29024 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:11.707777 29024 webserver.cc:460] Webserver started at http://127.2.180.254:37017/ using document root <none> and password file <none>
I20260808 15:07:11.708429 29024 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:11.708530 29024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:11.708841 29024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:11.711548 29024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "c757b2356209451883fbd7411c80b5b4"
format_stamp: "Formatted at 2026-08-08 15:07:11 on dist-test-slave-jlzn"
server_key: "049cf94109f653d47b182d84a16601d1"
server_key_iv: "d945e537c71e3ce11c9b11a1cb5c8a56"
server_key_version: "encryptionkey@0"
I20260808 15:07:11.712354 29024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "c757b2356209451883fbd7411c80b5b4"
format_stamp: "Formatted at 2026-08-08 15:07:11 on dist-test-slave-jlzn"
server_key: "049cf94109f653d47b182d84a16601d1"
server_key_iv: "d945e537c71e3ce11c9b11a1cb5c8a56"
server_key_version: "encryptionkey@0"
I20260808 15:07:11.717103 29024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:07:11.720322 29047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:11.721498 29024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:07:11.721673 29024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "c757b2356209451883fbd7411c80b5b4"
format_stamp: "Formatted at 2026-08-08 15:07:11 on dist-test-slave-jlzn"
server_key: "049cf94109f653d47b182d84a16601d1"
server_key_iv: "d945e537c71e3ce11c9b11a1cb5c8a56"
server_key_version: "encryptionkey@0"
I20260808 15:07:11.721875 29024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:11.750733 29024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:11.751471 29024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:11.751747 29024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:11.769660 29024 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:35579
I20260808 15:07:11.769672 29118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:35579 every 8 connection(s)
I20260808 15:07:11.770915 29024 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:11.774047 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 29024
I20260808 15:07:11.774233 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:11.774748 29119 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:11.774827 2771 external_mini_cluster.cc:1428] Setting key 2eb6d36b23dc79fe513207ae8b4c2bfb
I20260808 15:07:11.787781 29122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:11.776639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49158 (local address 127.2.180.254:35579)
0808 15:07:11.777609 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:11.777620 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:11.777661 (+ 41us) server_negotiation.cc:407] Connection header received
0808 15:07:11.779301 (+ 1640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:11.779325 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:11.779756 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:11.780107 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:11.780652 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:11.781942 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:11.783761 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:11.784433 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:11.784561 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:11.785740 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:11.785750 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:11.785762 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:11.786267 (+ 505us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:11.786336 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:11.786359 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:11.786440 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:11.786656 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 15:07:11.786926 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":372,"threads_started":1}
I20260808 15:07:11.787876 29119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4: Bootstrap starting.
I20260808 15:07:11.791159 29119 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:11.792140 29119 log.cc:824] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:11.794574 29119 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4: No bootstrap required, opened a new log
I20260808 15:07:11.801553 29119 raft_consensus.cc:348] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c757b2356209451883fbd7411c80b5b4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35579 } }
I20260808 15:07:11.801849 29119 raft_consensus.cc:374] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:11.801949 29119 raft_consensus.cc:729] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c757b2356209451883fbd7411c80b5b4, State: Initialized, Role: FOLLOWER
I20260808 15:07:11.802430 29119 consensus_queue.cc:260] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c757b2356209451883fbd7411c80b5b4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35579 } }
I20260808 15:07:11.802637 29119 raft_consensus.cc:388] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:11.802763 29119 raft_consensus.cc:482] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:11.802907 29119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:11.804926 29119 raft_consensus.cc:504] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c757b2356209451883fbd7411c80b5b4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35579 } }
I20260808 15:07:11.805387 29119 leader_election.cc:302] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c757b2356209451883fbd7411c80b5b4; no voters:
I20260808 15:07:11.805948 29119 leader_election.cc:288] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:11.806044 29124 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:11.806739 29124 raft_consensus.cc:686] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [term 1 LEADER]: Becoming Leader. State: Replica: c757b2356209451883fbd7411c80b5b4, State: Running, Role: LEADER
I20260808 15:07:11.807250 29124 consensus_queue.cc:237] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c757b2356209451883fbd7411c80b5b4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35579 } }
I20260808 15:07:11.807724 29119 sys_catalog.cc:564] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:11.809849 29126 sys_catalog.cc:455] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c757b2356209451883fbd7411c80b5b4. Latest consensus state: current_term: 1 leader_uuid: "c757b2356209451883fbd7411c80b5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c757b2356209451883fbd7411c80b5b4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35579 } } }
I20260808 15:07:11.810297 29126 sys_catalog.cc:458] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:11.810424 29125 sys_catalog.cc:455] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c757b2356209451883fbd7411c80b5b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c757b2356209451883fbd7411c80b5b4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35579 } } }
I20260808 15:07:11.810724 29125 sys_catalog.cc:458] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:11.811472 29140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:11.815507 29140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:11.823612 29140 catalog_manager.cc:1347] Generated new cluster ID: 38e88b5b0e5d4c7689932487eeb4cd5f
I20260808 15:07:11.823712 29140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:11.867996 29140 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:11.869743 29140 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:11.889786 29140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4: Generated new TSK 0
I20260808 15:07:11.890524 29140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:11.894577 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35579
--builtin_ntp_servers=127.2.180.212:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:07:12.214476 29151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:12.214735 29151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:12.214859 29151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:12.221771 29151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:12.221859 29151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:12.221993 29151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:07:12.230743 29151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:12.231863 29151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:12.233506 29151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:12.245281 29158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:12.245332 29160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:12.245421 29167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:12.246237 29151 server_base.cc:1034] running on GCE node
I20260808 15:07:12.246788 29151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:12.247603 29151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:12.248797 29151 hybrid_clock.cc:648] HybridClock initialized: now 1786201632248737 us; error 47 us; skew 500 ppm
I20260808 15:07:12.249099 29151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:12.251156 29151 webserver.cc:460] Webserver started at http://127.2.180.193:46427/ using document root <none> and password file <none>
I20260808 15:07:12.251796 29151 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:12.251940 29151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:12.252241 29151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:12.254855 29151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "87f8da81fb234d10b2dfd10007c47eba"
format_stamp: "Formatted at 2026-08-08 15:07:12 on dist-test-slave-jlzn"
server_key: "fb553083065f17a4ff8509a235d664e8"
server_key_iv: "679267f63ac6a3f8b753bdaacb12845c"
server_key_version: "encryptionkey@0"
I20260808 15:07:12.255595 29151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "87f8da81fb234d10b2dfd10007c47eba"
format_stamp: "Formatted at 2026-08-08 15:07:12 on dist-test-slave-jlzn"
server_key: "fb553083065f17a4ff8509a235d664e8"
server_key_iv: "679267f63ac6a3f8b753bdaacb12845c"
server_key_version: "encryptionkey@0"
I20260808 15:07:12.260249 29151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 15:07:12.263331 29175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:12.264510 29151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260808 15:07:12.264670 29151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "87f8da81fb234d10b2dfd10007c47eba"
format_stamp: "Formatted at 2026-08-08 15:07:12 on dist-test-slave-jlzn"
server_key: "fb553083065f17a4ff8509a235d664e8"
server_key_iv: "679267f63ac6a3f8b753bdaacb12845c"
server_key_version: "encryptionkey@0"
I20260808 15:07:12.264884 29151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:12.278981 29151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:12.279655 29151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:12.279999 29151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:12.280848 29151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:12.282114 29151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:12.282207 29151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:12.282311 29151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:12.282380 29151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:12.311829 29151 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:37483
I20260808 15:07:12.311882 29340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:37483 every 8 connection(s)
I20260808 15:07:12.312989 29151 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:07:12.320863 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 29151
I20260808 15:07:12.321007 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:07:12.321426 2771 external_mini_cluster.cc:1428] Setting key d17f1aa92c753d8ed5af23881ffc4ec2
I20260808 15:07:12.324241 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35579
--builtin_ntp_servers=127.2.180.212:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:07:12.324329 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:12.315180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:40619 (local address 127.2.180.254:35579)
0808 15:07:12.315578 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:12.315584 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:12.316697 (+ 1113us) server_negotiation.cc:407] Connection header received
0808 15:07:12.318333 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:12.318338 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:12.318407 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:12.318499 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:12.319400 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:12.320128 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:12.321665 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:12.322155 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:12.322256 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:12.323543 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:12.323548 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:12.323551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:12.323567 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:12.323605 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:12.323617 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:12.323697 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:12.323907 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 15:07:12.324002 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":178,"threads_started":1}
I20260808 15:07:12.325276 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:12.315724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35579 (local address 127.2.180.193:40619)
0808 15:07:12.316556 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:12.316599 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:12.318099 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:12.318680 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:12.318687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:12.318765 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:12.319270 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:12.319285 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:12.320245 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:12.320249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:12.321509 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:12.321522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:12.322366 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:12.322374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:12.322569 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:12.323251 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:12.323255 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:12.323305 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:12.323374 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:12.323703 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:12.323714 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:12.323722 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:07:12.323896 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 15:07:12.324248 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":252,"threads_started":1}
I20260808 15:07:12.326722 29342 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35579
I20260808 15:07:12.327001 29342 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:12.327793 29342 heartbeater.cc:511] Master 127.2.180.254:35579 requested a full tablet report, sending...
I20260808 15:07:12.329862 29067 ts_manager.cc:194] Registered new tserver with Master: 87f8da81fb234d10b2dfd10007c47eba (127.2.180.193:37483)
I20260808 15:07:12.331712 29067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.193:40619
W20260808 15:07:12.656880 29347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:12.657119 29347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:12.657238 29347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:12.664165 29347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:12.664247 29347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:12.664386 29347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:07:12.672996 29347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:12.674121 29347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:12.675818 29347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:12.687889 29354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:12.688020 29357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:12.688032 29355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:12.689013 29347 server_base.cc:1034] running on GCE node
I20260808 15:07:12.689543 29347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:12.690363 29347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:12.691561 29347 hybrid_clock.cc:648] HybridClock initialized: now 1786201632691506 us; error 48 us; skew 500 ppm
I20260808 15:07:12.691872 29347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:12.693841 29347 webserver.cc:460] Webserver started at http://127.2.180.194:38991/ using document root <none> and password file <none>
I20260808 15:07:12.694451 29347 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:12.694550 29347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:12.694844 29347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:12.697458 29347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "4825879a750f46779cd06d96e958e02e"
format_stamp: "Formatted at 2026-08-08 15:07:12 on dist-test-slave-jlzn"
server_key: "081753b7335cf3f59cc6d2d7f63a1013"
server_key_iv: "6e0928de977cf7123b36f31019d3c15c"
server_key_version: "encryptionkey@0"
I20260808 15:07:12.698225 29347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "4825879a750f46779cd06d96e958e02e"
format_stamp: "Formatted at 2026-08-08 15:07:12 on dist-test-slave-jlzn"
server_key: "081753b7335cf3f59cc6d2d7f63a1013"
server_key_iv: "6e0928de977cf7123b36f31019d3c15c"
server_key_version: "encryptionkey@0"
I20260808 15:07:12.702909 29347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 15:07:12.706030 29366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:12.707135 29347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 15:07:12.707288 29347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "4825879a750f46779cd06d96e958e02e"
format_stamp: "Formatted at 2026-08-08 15:07:12 on dist-test-slave-jlzn"
server_key: "081753b7335cf3f59cc6d2d7f63a1013"
server_key_iv: "6e0928de977cf7123b36f31019d3c15c"
server_key_version: "encryptionkey@0"
I20260808 15:07:12.707470 29347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:12.730280 29347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:12.730962 29347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:12.731233 29347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:12.732118 29347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:12.733428 29347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:12.733522 29347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:12.733623 29347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:12.733695 29347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:12.762519 29347 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:41113
I20260808 15:07:12.762590 29531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:41113 every 8 connection(s)
I20260808 15:07:12.763842 29347 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:07:12.770779 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 29347
I20260808 15:07:12.771098 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:07:12.772138 2771 external_mini_cluster.cc:1428] Setting key 223d799d1976d9dfb6ecf8fddc103a39
I20260808 15:07:12.774955 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35579
--builtin_ntp_servers=127.2.180.212:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:07:12.777972 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:12.766111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:47251 (local address 127.2.180.254:35579)
0808 15:07:12.766273 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:12.766277 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:12.767494 (+ 1217us) server_negotiation.cc:407] Connection header received
0808 15:07:12.769945 (+ 2451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:12.769951 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:12.770031 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:12.770104 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:12.771586 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:12.772773 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:12.775023 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:12.775589 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:12.775710 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:12.777223 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:12.777228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:12.777232 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:12.777252 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:12.777292 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:12.777307 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:12.777393 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:12.777587 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 15:07:12.777679 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:07:12.779011 29538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:12.766530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35579 (local address 127.2.180.194:47251)
0808 15:07:12.767350 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:12.767394 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:12.769636 (+ 2242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:12.770585 (+ 949us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:12.770595 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:12.770700 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:12.771367 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:12.771389 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:12.772976 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:12.772981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:12.774863 (+ 1882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:12.774878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:12.775844 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:12.775852 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:12.776114 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:12.776856 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:12.776860 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:12.776912 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:12.777020 (+ 108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:12.777395 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:12.777403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:12.777411 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:07:12.777558 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:07:12.777899 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":252,"threads_started":1}
I20260808 15:07:12.780603 29535 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35579
I20260808 15:07:12.780889 29535 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:12.781672 29535 heartbeater.cc:511] Master 127.2.180.254:35579 requested a full tablet report, sending...
I20260808 15:07:12.783227 29067 ts_manager.cc:194] Registered new tserver with Master: 4825879a750f46779cd06d96e958e02e (127.2.180.194:41113)
I20260808 15:07:12.784245 29067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.194:47251
W20260808 15:07:13.096271 29540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:13.096511 29540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:13.096607 29540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:13.103582 29540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:13.103669 29540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:13.103776 29540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:07:13.112501 29540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:13.113550 29540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:13.115154 29540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:13.127351 29548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:13.127316 29549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:13.127386 29553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:13.127874 29540 server_base.cc:1034] running on GCE node
I20260808 15:07:13.128545 29540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:13.129380 29540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:13.130573 29540 hybrid_clock.cc:648] HybridClock initialized: now 1786201633130535 us; error 30 us; skew 500 ppm
I20260808 15:07:13.130846 29540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:13.132853 29540 webserver.cc:460] Webserver started at http://127.2.180.195:44763/ using document root <none> and password file <none>
I20260808 15:07:13.133472 29540 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:13.133569 29540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:13.133867 29540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:13.136538 29540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "c5fb346942104debbe98e82fccf9b0d3"
format_stamp: "Formatted at 2026-08-08 15:07:13 on dist-test-slave-jlzn"
server_key: "7b62b0418766286d4580fa17e879a1c4"
server_key_iv: "f28942e3159a4fe8e6522bf63bd2d6f1"
server_key_version: "encryptionkey@0"
I20260808 15:07:13.137284 29540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "c5fb346942104debbe98e82fccf9b0d3"
format_stamp: "Formatted at 2026-08-08 15:07:13 on dist-test-slave-jlzn"
server_key: "7b62b0418766286d4580fa17e879a1c4"
server_key_iv: "f28942e3159a4fe8e6522bf63bd2d6f1"
server_key_version: "encryptionkey@0"
I20260808 15:07:13.141853 29540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260808 15:07:13.144949 29560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:13.146116 29540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:07:13.146272 29540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "c5fb346942104debbe98e82fccf9b0d3"
format_stamp: "Formatted at 2026-08-08 15:07:13 on dist-test-slave-jlzn"
server_key: "7b62b0418766286d4580fa17e879a1c4"
server_key_iv: "f28942e3159a4fe8e6522bf63bd2d6f1"
server_key_version: "encryptionkey@0"
I20260808 15:07:13.146454 29540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:13.164016 29540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:13.164722 29540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:13.164989 29540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:13.165846 29540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:13.167218 29540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:13.167310 29540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:13.167411 29540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:13.167480 29540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:13.195952 29540 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:32939
I20260808 15:07:13.196014 29726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:32939 every 8 connection(s)
I20260808 15:07:13.197161 29540 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:07:13.202589 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 29540
I20260808 15:07:13.202762 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:07:13.203208 2771 external_mini_cluster.cc:1428] Setting key 51489a6bad4c02476faad03dc2538bee
I20260808 15:07:13.209252 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.199567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:33021 (local address 127.2.180.254:35579)
0808 15:07:13.199776 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.199780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.201099 (+ 1319us) server_negotiation.cc:407] Connection header received
0808 15:07:13.202829 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.202834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.202921 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.203009 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:13.204185 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.205145 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.206676 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.207099 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.207208 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.208529 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:13.208534 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:13.208537 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:13.208554 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:13.208587 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:13.208599 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:13.208687 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.208911 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.208993 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:07:13.210196 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.200094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35579 (local address 127.2.180.195:33021)
0808 15:07:13.200924 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.200965 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:13.202559 (+ 1594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:13.203251 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:13.203258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:13.203360 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:13.204017 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.204044 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.205297 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.205301 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.206541 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.206553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.207308 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.207316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.207546 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.208228 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:13.208231 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:13.208267 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:13.208344 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:13.208703 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:13.208713 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:13.208721 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:07:13.208857 (+ 136us) client_negotiation.cc:240] Negotiation successful
0808 15:07:13.209131 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":249,"threads_started":1}
I20260808 15:07:13.211490 29728 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35579
I20260808 15:07:13.211766 29728 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:13.212507 29728 heartbeater.cc:511] Master 127.2.180.254:35579 requested a full tablet report, sending...
I20260808 15:07:13.213814 29067 ts_manager.cc:194] Registered new tserver with Master: c5fb346942104debbe98e82fccf9b0d3 (127.2.180.195:32939)
I20260808 15:07:13.214581 29067 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.195:33021
I20260808 15:07:13.217870 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:07:13.227221 29344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.221812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49174 (local address 127.2.180.254:35579)
0808 15:07:13.221950 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.221955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.222336 (+ 381us) server_negotiation.cc:407] Connection header received
0808 15:07:13.222497 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.222501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.222557 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.222628 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:13.223079 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.223733 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.225147 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.225547 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.225659 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.226600 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:13.226605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:13.226608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:13.226624 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:13.226663 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:13.226676 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:13.226749 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.226881 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.226965 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 15:07:13.230518 29067 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49174:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:07:13.236963 29067 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:07:13.264344 29747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.247674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:32939 (local address 127.0.0.1:51632)
0808 15:07:13.248845 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.248885 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:13.249071 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:13.256617 (+ 7546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:13.256625 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:13.256666 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:13.257034 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.257046 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.259609 (+ 2563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.259615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.261553 (+ 1938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.261568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.263466 (+ 1898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.263472 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.263676 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.263729 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 15:07:13.263874 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 15:07:13.264043 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":948,"threads_started":1}
I20260808 15:07:13.265192 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.247680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51632 (local address 127.2.180.195:32939)
0808 15:07:13.248437 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.248445 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.249260 (+ 815us) server_negotiation.cc:407] Connection header received
0808 15:07:13.255673 (+ 6413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.255682 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.255945 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.256124 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:13.257840 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.259357 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.261719 (+ 2362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.262937 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.263061 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.263337 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.264724 (+ 1387us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.264874 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":477,"threads_started":1}
I20260808 15:07:13.269024 29748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.249384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52488)
0808 15:07:13.261898 (+ 12514us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.261914 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:13.262019 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:13.262624 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:13.262632 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:13.262678 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:13.263169 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.263181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.265129 (+ 1948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.265134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.266940 (+ 1806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.266950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.268325 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.268331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.268489 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.268516 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 15:07:13.268566 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 15:07:13.268663 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10767,"mutex_wait_us":39,"spinlock_wait_cycles":3584,"thread_start_us":9940,"threads_started":1}
I20260808 15:07:13.269043 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.248543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.2.180.194:41113)
0808 15:07:13.260404 (+ 11861us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.260412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.261998 (+ 1586us) server_negotiation.cc:407] Connection header received
0808 15:07:13.262193 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.262199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.262351 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.262503 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:13.263141 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.264421 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.266718 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.268037 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.268232 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.268486 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.268561 (+ 75us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.268711 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11705,"thread_start_us":13129,"threads_started":1}
I20260808 15:07:13.268582 29632 tablet_service.cc:1467] Processing CreateTablet for tablet b1efbb59f6e440898663acc4666e5f8e (DEFAULT_TABLE table=test-table [id=59a6fc2fbd9b4f40af9877406f2e3580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:13.269753 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.248143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41113 (local address 127.0.0.1:47318)
0808 15:07:13.261871 (+ 13728us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.261891 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:13.262020 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:13.262513 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:13.262518 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:13.262541 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:13.262921 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.262934 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.264581 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.264586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.266518 (+ 1932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.266533 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.268234 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.268241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.268420 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.268453 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:07:13.268527 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 15:07:13.268657 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11318,"mutex_wait_us":3,"spinlock_wait_cycles":9600,"thread_start_us":11855,"threads_started":1}
I20260808 15:07:13.270224 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.249709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.2.180.193:37483)
0808 15:07:13.260022 (+ 10313us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.260031 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.262106 (+ 2075us) server_negotiation.cc:407] Connection header received
0808 15:07:13.262208 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.262215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.262413 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.262972 (+ 559us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:13.263504 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.264987 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.267102 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.268117 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.269432 (+ 1315us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.269646 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.269723 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.269876 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10145,"thread_start_us":10225,"threads_started":1}
I20260808 15:07:13.270289 29632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1efbb59f6e440898663acc4666e5f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:13.273049 29246 tablet_service.cc:1467] Processing CreateTablet for tablet b1efbb59f6e440898663acc4666e5f8e (DEFAULT_TABLE table=test-table [id=59a6fc2fbd9b4f40af9877406f2e3580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:13.273310 29441 tablet_service.cc:1467] Processing CreateTablet for tablet b1efbb59f6e440898663acc4666e5f8e (DEFAULT_TABLE table=test-table [id=59a6fc2fbd9b4f40af9877406f2e3580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:13.274132 29246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1efbb59f6e440898663acc4666e5f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:13.274614 29441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1efbb59f6e440898663acc4666e5f8e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:13.285401 29756 tablet_bootstrap.cc:492] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba: Bootstrap starting.
I20260808 15:07:13.288270 29756 tablet_bootstrap.cc:654] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:13.288543 29757 tablet_bootstrap.cc:492] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: Bootstrap starting.
I20260808 15:07:13.289690 29756 log.cc:824] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:13.289916 29758 tablet_bootstrap.cc:492] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: Bootstrap starting.
I20260808 15:07:13.292414 29757 tablet_bootstrap.cc:654] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:13.292605 29758 tablet_bootstrap.cc:654] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:13.293488 29758 log.cc:824] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:13.293555 29757 log.cc:824] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:13.294461 29756 tablet_bootstrap.cc:492] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba: No bootstrap required, opened a new log
I20260808 15:07:13.295176 29756 ts_tablet_manager.cc:1397] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260808 15:07:13.296334 29757 tablet_bootstrap.cc:492] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: No bootstrap required, opened a new log
I20260808 15:07:13.296388 29758 tablet_bootstrap.cc:492] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: No bootstrap required, opened a new log
I20260808 15:07:13.296558 29757 ts_tablet_manager.cc:1397] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260808 15:07:13.296654 29758 ts_tablet_manager.cc:1397] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:07:13.303233 29757 raft_consensus.cc:348] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.303556 29757 raft_consensus.cc:374] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:13.303648 29757 raft_consensus.cc:729] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5fb346942104debbe98e82fccf9b0d3, State: Initialized, Role: FOLLOWER
I20260808 15:07:13.304220 29757 consensus_queue.cc:260] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.305209 29728 heartbeater.cc:503] Master 127.2.180.254:35579 was elected leader, sending a full tablet report...
I20260808 15:07:13.305663 29757 ts_tablet_manager.cc:1428] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260808 15:07:13.306284 29758 raft_consensus.cc:348] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.306699 29758 raft_consensus.cc:374] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:13.306818 29758 raft_consensus.cc:729] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4825879a750f46779cd06d96e958e02e, State: Initialized, Role: FOLLOWER
I20260808 15:07:13.306922 29756 raft_consensus.cc:348] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.307336 29756 raft_consensus.cc:374] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:13.307446 29756 raft_consensus.cc:729] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87f8da81fb234d10b2dfd10007c47eba, State: Initialized, Role: FOLLOWER
I20260808 15:07:13.307391 29758 consensus_queue.cc:260] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.308198 29756 consensus_queue.cc:260] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.309406 29342 heartbeater.cc:503] Master 127.2.180.254:35579 was elected leader, sending a full tablet report...
I20260808 15:07:13.310148 29756 ts_tablet_manager.cc:1428] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba: Time spent starting tablet: real 0.015s user 0.006s sys 0.007s
I20260808 15:07:13.310994 29535 heartbeater.cc:503] Master 127.2.180.254:35579 was elected leader, sending a full tablet report...
I20260808 15:07:13.311910 29758 ts_tablet_manager.cc:1428] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260808 15:07:13.315013 29343 tablet.cc:2369] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:07:13.342492 29766 raft_consensus.cc:482] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:07:13.342787 29766 raft_consensus.cc:504] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.344429 29766 leader_election.cc:288] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87f8da81fb234d10b2dfd10007c47eba (127.2.180.193:37483), 4825879a750f46779cd06d96e958e02e (127.2.180.194:41113)
I20260808 15:07:13.349792 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.345061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:50579 (local address 127.2.180.193:37483)
0808 15:07:13.345238 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.345243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.345257 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:07:13.345309 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.345312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.345372 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.345449 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:13.345869 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.346539 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.348145 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.348921 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.349087 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.349155 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.349412 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.349544 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:07:13.349805 29731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.344551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.2.180.195:50579)
0808 15:07:13.344841 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.344861 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:13.344987 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:13.345531 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:13.345537 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:13.345566 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:13.345729 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.345738 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.346666 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.346670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.347993 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.348005 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.349047 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.349053 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.349249 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.349286 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 15:07:13.349379 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 15:07:13.349516 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":2}
I20260808 15:07:13.350765 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.344992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:52899 (local address 127.2.180.194:41113)
0808 15:07:13.345187 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.345193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.345727 (+ 534us) server_negotiation.cc:407] Connection header received
0808 15:07:13.345891 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.345897 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.345950 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.346023 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:13.346518 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.347185 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.348896 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.349995 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.350165 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.350228 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.350459 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.350569 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 15:07:13.350802 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.344811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:41113 (local address 127.2.180.195:52899)
0808 15:07:13.345618 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.345632 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:13.345766 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:13.346053 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:13.346057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:13.346080 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:13.346375 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.346383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.347332 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.347335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.348697 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:13.348709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.350186 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.350191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:13.350304 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.350332 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 15:07:13.350424 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 15:07:13.350538 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":696,"thread_start_us":521,"threads_started":1}
I20260808 15:07:13.350884 29277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1efbb59f6e440898663acc4666e5f8e" candidate_uuid: "c5fb346942104debbe98e82fccf9b0d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87f8da81fb234d10b2dfd10007c47eba" is_pre_election: true
I20260808 15:07:13.351356 29277 raft_consensus.cc:2454] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5fb346942104debbe98e82fccf9b0d3 in term 0.
I20260808 15:07:13.351792 29472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1efbb59f6e440898663acc4666e5f8e" candidate_uuid: "c5fb346942104debbe98e82fccf9b0d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4825879a750f46779cd06d96e958e02e" is_pre_election: true
I20260808 15:07:13.352038 29561 leader_election.cc:302] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87f8da81fb234d10b2dfd10007c47eba, c5fb346942104debbe98e82fccf9b0d3; no voters:
I20260808 15:07:13.352339 29472 raft_consensus.cc:2454] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c5fb346942104debbe98e82fccf9b0d3 in term 0.
I20260808 15:07:13.352401 29766 raft_consensus.cc:2781] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:07:13.352507 29766 raft_consensus.cc:482] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:07:13.352600 29766 raft_consensus.cc:3037] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:13.354699 29766 raft_consensus.cc:504] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.355214 29766 leader_election.cc:288] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [CANDIDATE]: Term 1 election: Requested vote from peers 87f8da81fb234d10b2dfd10007c47eba (127.2.180.193:37483), 4825879a750f46779cd06d96e958e02e (127.2.180.194:41113)
I20260808 15:07:13.355610 29472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1efbb59f6e440898663acc4666e5f8e" candidate_uuid: "c5fb346942104debbe98e82fccf9b0d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4825879a750f46779cd06d96e958e02e"
I20260808 15:07:13.355634 29277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1efbb59f6e440898663acc4666e5f8e" candidate_uuid: "c5fb346942104debbe98e82fccf9b0d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87f8da81fb234d10b2dfd10007c47eba"
I20260808 15:07:13.355762 29472 raft_consensus.cc:3037] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:13.355819 29277 raft_consensus.cc:3037] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:13.357878 29472 raft_consensus.cc:2454] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5fb346942104debbe98e82fccf9b0d3 in term 1.
I20260808 15:07:13.357934 29277 raft_consensus.cc:2454] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c5fb346942104debbe98e82fccf9b0d3 in term 1.
I20260808 15:07:13.358247 29564 leader_election.cc:302] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4825879a750f46779cd06d96e958e02e, c5fb346942104debbe98e82fccf9b0d3; no voters:
I20260808 15:07:13.358553 29766 raft_consensus.cc:2781] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:13.358915 29766 raft_consensus.cc:686] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 1 LEADER]: Becoming Leader. State: Replica: c5fb346942104debbe98e82fccf9b0d3, State: Running, Role: LEADER
I20260808 15:07:13.359387 29766 consensus_queue.cc:237] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } }
I20260808 15:07:13.363360 29067 catalog_manager.cc:5495] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to c5fb346942104debbe98e82fccf9b0d3 (127.2.180.195). New cstate: current_term: 1 leader_uuid: "c5fb346942104debbe98e82fccf9b0d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5fb346942104debbe98e82fccf9b0d3" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 32939 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 } health_report { overall_health: UNKNOWN } } }
I20260808 15:07:13.378444 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.374135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51648 (local address 127.2.180.195:32939)
0808 15:07:13.374282 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.374286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.374300 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:07:13.374365 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.374369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.374430 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.374541 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:07:13.374802 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.375497 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.376820 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.377208 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.377293 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.377653 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:07:13.378085 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:07:13.378189 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.378227 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.378280 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:07:13.385574 29277 raft_consensus.cc:1261] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba [term 1 FOLLOWER]: Refusing update from remote peer c5fb346942104debbe98e82fccf9b0d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:13.385577 29472 raft_consensus.cc:1261] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 1 FOLLOWER]: Refusing update from remote peer c5fb346942104debbe98e82fccf9b0d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:13.386394 29772 consensus_queue.cc:1035] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87f8da81fb234d10b2dfd10007c47eba" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 37483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:13.386715 29766 consensus_queue.cc:1035] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4825879a750f46779cd06d96e958e02e" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 41113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:13.396221 29777 mvcc.cc:205] Tried to move back new op lower bound from 7316281890338590720 to 7316281890243956736. Current Snapshot: MvccSnapshot[applied={T|T < 7316281890338590720}]
I20260808 15:07:13.407112 29066 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:13.407358 29066 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49174:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:13.410660 29066 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c757b2356209451883fbd7411c80b5b4: Sending DeleteTablet for 3 replicas of tablet b1efbb59f6e440898663acc4666e5f8e
I20260808 15:07:13.411785 29632 tablet_service.cc:1514] Processing DeleteTablet for tablet b1efbb59f6e440898663acc4666e5f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:13 UTC) from {username='slave'} at 127.0.0.1:51632
I20260808 15:07:13.412060 29441 tablet_service.cc:1514] Processing DeleteTablet for tablet b1efbb59f6e440898663acc4666e5f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:13 UTC) from {username='slave'} at 127.0.0.1:47318
I20260808 15:07:13.411780 29246 tablet_service.cc:1514] Processing DeleteTablet for tablet b1efbb59f6e440898663acc4666e5f8e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:13 UTC) from {username='slave'} at 127.0.0.1:52488
I20260808 15:07:13.414179 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 29151
I20260808 15:07:13.424153 29789 tablet_replica.cc:330] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: stopping tablet replica
I20260808 15:07:13.424172 29754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.415279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52504 (local address 127.2.180.193:37483)
0808 15:07:13.415463 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.415470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.416148 (+ 678us) server_negotiation.cc:407] Connection header received
0808 15:07:13.416403 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.416409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.416504 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.416582 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:13.417247 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.418376 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.420310 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.420940 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.421046 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.422492 (+ 1446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:13.422503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:13.422519 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:13.423266 (+ 747us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:13.423329 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:13.423354 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:13.423435 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.423683 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.423792 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:07:13.425024 29789 raft_consensus.cc:2229] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:07:13.425195 29335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:13.425721 29789 raft_consensus.cc:2258] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:13.428817 29791 tablet_replica.cc:330] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: stopping tablet replica
I20260808 15:07:13.429325 29791 raft_consensus.cc:2229] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:13.429493 29789 ts_tablet_manager.cc:1905] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:13.429796 29791 raft_consensus.cc:2258] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:13.433513 29791 ts_tablet_manager.cc:1905] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:13.435782 29789 ts_tablet_manager.cc:1918] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:13.436023 29789 log.cc:1196] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/b1efbb59f6e440898663acc4666e5f8e
I20260808 15:07:13.436638 29789 ts_tablet_manager.cc:1939] T b1efbb59f6e440898663acc4666e5f8e P c5fb346942104debbe98e82fccf9b0d3: Deleting consensus metadata
I20260808 15:07:13.438473 29048 catalog_manager.cc:4841] TS c5fb346942104debbe98e82fccf9b0d3 (127.2.180.195:32939): tablet b1efbb59f6e440898663acc4666e5f8e (table test-table [id=59a6fc2fbd9b4f40af9877406f2e3580]) successfully deleted
I20260808 15:07:13.439699 29791 ts_tablet_manager.cc:1918] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:13.439851 29791 log.cc:1196] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/b1efbb59f6e440898663acc4666e5f8e
I20260808 15:07:13.440270 29791 ts_tablet_manager.cc:1939] T b1efbb59f6e440898663acc4666e5f8e P 4825879a750f46779cd06d96e958e02e: Deleting consensus metadata
I20260808 15:07:13.441546 29049 catalog_manager.cc:4841] TS 4825879a750f46779cd06d96e958e02e (127.2.180.194:41113): tablet b1efbb59f6e440898663acc4666e5f8e (table test-table [id=59a6fc2fbd9b4f40af9877406f2e3580]) successfully deleted
I20260808 15:07:13.552739 29792 tablet_replica.cc:330] T b1efbb59f6e440898663acc4666e5f8e P 87f8da81fb234d10b2dfd10007c47eba: stopping tablet replica
I20260808 15:07:13.552909 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 29151
I20260808 15:07:13.573765 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 29347
W20260808 15:07:13.573748 29048 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.180.193:37483 (error 108)
I20260808 15:07:13.575824 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.575308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52506)
0808 15:07:13.575477 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.575620 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 15:07:13.576119 29048 catalog_manager.cc:4584] TS 87f8da81fb234d10b2dfd10007c47eba (127.2.180.193:37483): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1efbb59f6e440898663acc4666e5f8e: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
I20260808 15:07:13.582980 29749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.574910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47332 (local address 127.2.180.194:41113)
0808 15:07:13.575082 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.575088 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.575107 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:07:13.575174 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.575180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.575283 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.575387 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:13.576009 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.576885 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.578884 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.579529 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.579669 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.581126 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:13.581138 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:13.581156 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:13.581933 (+ 777us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:13.581996 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:13.582022 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:13.582131 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.582366 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.582489 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:07:13.583688 29527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:13.629715 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.629280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52522)
0808 15:07:13.629470 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.629555 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 15:07:13.630748 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.630372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52532)
0808 15:07:13.630522 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.630628 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 15:07:13.691404 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.690784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52536)
0808 15:07:13.691069 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.691186 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260808 15:07:13.692718 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.692310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52548)
0808 15:07:13.692487 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.692580 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 15:07:13.699601 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 29347
I20260808 15:07:13.721535 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 29540
I20260808 15:07:13.734335 29746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.722418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51664 (local address 127.2.180.195:32939)
0808 15:07:13.722644 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:13.722651 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:13.722676 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:07:13.722845 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:13.722851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:13.722950 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:13.723032 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:13.723515 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.724384 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.726278 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:13.726734 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:13.726829 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:13.732885 (+ 6056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:13.732894 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:13.732909 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:13.733574 (+ 665us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:13.733632 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:13.733656 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:13.733734 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:13.733904 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 15:07:13.733986 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:07:13.735052 29720 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:13.773469 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.773041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52554)
0808 15:07:13.773203 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.773290 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 15:07:13.774609 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.774219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52558)
0808 15:07:13.774387 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.774463 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 15:07:13.853509 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 29540
I20260808 15:07:13.880728 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 29024
I20260808 15:07:13.881825 29111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:13.985469 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 29024
I20260808 15:07:13.985499 29752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:13.984954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:37483 (local address 127.0.0.1:52572)
0808 15:07:13.985180 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:13.985303 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:37483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
2026-08-08T15:07:14Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2710 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:14 dist-test-slave-jlzn krb5kdc[29804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:14 dist-test-slave-jlzn krb5kdc[29804](info): set up 2 sockets
Aug 08 15:07:14 dist-test-slave-jlzn krb5kdc[29804](info): commencing operation
krb5kdc: starting...
W20260808 15:07:15.170841 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:15 dist-test-slave-jlzn krb5kdc[29804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:07:15.404836 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35465
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:35465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:15.721150 29822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:15.721431 29822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:15.721566 29822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:15.728402 29822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:15.728508 29822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:15.728587 29822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:15.728669 29822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:15.728731 29822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:15.737905 29822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:35465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:15.739084 29822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:15.740350 29822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:15.751410 29827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:15.751361 29829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:15.751333 29833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:15.752410 29822 server_base.cc:1034] running on GCE node
I20260808 15:07:15.753150 29822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:15.754392 29822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:15.755596 29822 hybrid_clock.cc:648] HybridClock initialized: now 1786201635755547 us; error 34 us; skew 500 ppm
I20260808 15:07:15.755956 29822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:15 dist-test-slave-jlzn krb5kdc[29804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201635, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:15.758836 29822 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:07:15.760298 29822 webserver.cc:460] Webserver started at http://127.2.180.254:39775/ using document root <none> and password file <none>
I20260808 15:07:15.760915 29822 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:15.761034 29822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:15.761329 29822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:15.764212 29822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "9b2555ec1d164bd484f4b3e97ac67fb7"
format_stamp: "Formatted at 2026-08-08 15:07:15 on dist-test-slave-jlzn"
server_key: "72816af10440cafbeb2f2556e97db09f"
server_key_iv: "c8075029ff56fdaa0d297c4c59993f36"
server_key_version: "encryptionkey@0"
I20260808 15:07:15.764922 29822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "9b2555ec1d164bd484f4b3e97ac67fb7"
format_stamp: "Formatted at 2026-08-08 15:07:15 on dist-test-slave-jlzn"
server_key: "72816af10440cafbeb2f2556e97db09f"
server_key_iv: "c8075029ff56fdaa0d297c4c59993f36"
server_key_version: "encryptionkey@0"
I20260808 15:07:15.769750 29822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:07:15.772970 29842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:15.774128 29822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:07:15.774289 29822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "9b2555ec1d164bd484f4b3e97ac67fb7"
format_stamp: "Formatted at 2026-08-08 15:07:15 on dist-test-slave-jlzn"
server_key: "72816af10440cafbeb2f2556e97db09f"
server_key_iv: "c8075029ff56fdaa0d297c4c59993f36"
server_key_version: "encryptionkey@0"
I20260808 15:07:15.774456 29822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:15.802172 29822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:15.806209 29822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:15.806473 29822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:15.824056 29822 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:35465
I20260808 15:07:15.824088 29925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:35465 every 8 connection(s)
I20260808 15:07:15.825325 29822 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:15.828961 29926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:15.835423 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 29822
I20260808 15:07:15.835593 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:15.836027 2771 external_mini_cluster.cc:1428] Setting key 58ab40db2e6ae0d1c1050f7cc3579ab5
I20260808 15:07:15.839376 29926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7: Bootstrap starting.
I20260808 15:07:15.842290 29926 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:15.843626 29926 log.cc:824] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:15.846628 29926 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7: No bootstrap required, opened a new log
Aug 08 15:07:15 dist-test-slave-jlzn krb5kdc[29804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:15.855065 29926 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2555ec1d164bd484f4b3e97ac67fb7" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35465 } }
I20260808 15:07:15.855540 29926 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:15.855656 29926 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b2555ec1d164bd484f4b3e97ac67fb7, State: Initialized, Role: FOLLOWER
I20260808 15:07:15.856293 29926 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2555ec1d164bd484f4b3e97ac67fb7" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35465 } }
I20260808 15:07:15.856545 29926 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:15.856698 29926 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:15.856915 29926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:15.858721 29930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:15.837826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.2.180.254:35465)
0808 15:07:15.838586 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:15.838597 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:15.838637 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:07:15.840373 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:15.840394 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:15.840758 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:15.841093 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:15.842547 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:15.844509 (+ 1962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:15.845907 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:15.846909 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:15.847121 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:15.851455 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:15.851463 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:15.851476 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:15.851527 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:15.854518 (+ 2991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:15.855352 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:15.855361 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:15.855367 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:15.855549 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:15.855993 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:15.855998 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:15.856000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:15.856579 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:15.856862 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:15.857293 (+ 431us) server_negotiation.cc:299] Negotiation successful
0808 15:07:15.857655 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":369,"threads_started":1}
I20260808 15:07:15.859422 29926 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2555ec1d164bd484f4b3e97ac67fb7" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35465 } }
I20260808 15:07:15.859917 29926 leader_election.cc:302] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b2555ec1d164bd484f4b3e97ac67fb7; no voters:
I20260808 15:07:15.860518 29926 leader_election.cc:288] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:15.860646 29932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:15.861955 29932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [term 1 LEADER]: Becoming Leader. State: Replica: 9b2555ec1d164bd484f4b3e97ac67fb7, State: Running, Role: LEADER
I20260808 15:07:15.862509 29932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2555ec1d164bd484f4b3e97ac67fb7" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35465 } }
I20260808 15:07:15.862712 29926 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:15.865418 29935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b2555ec1d164bd484f4b3e97ac67fb7. Latest consensus state: current_term: 1 leader_uuid: "9b2555ec1d164bd484f4b3e97ac67fb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2555ec1d164bd484f4b3e97ac67fb7" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35465 } } }
I20260808 15:07:15.865779 29935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:15.866640 29943 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:15.866742 29934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b2555ec1d164bd484f4b3e97ac67fb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b2555ec1d164bd484f4b3e97ac67fb7" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35465 } } }
I20260808 15:07:15.867033 29934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:15.871773 29943 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:15.881297 29943 catalog_manager.cc:1347] Generated new cluster ID: 8883fc8c72a843328235ce73edf56e37
I20260808 15:07:15.881389 29943 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:15.896355 29943 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:15.897485 29943 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:15.904330 29943 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7: Generated new TSK 0
I20260808 15:07:15.904990 29943 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:07:15.979730 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35465
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:16.303795 29960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:16.304101 29960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:16.304199 29960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:16.311002 29960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:16.311102 29960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:16.311213 29960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:07:16.320186 29960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:16.321231 29960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:16.322409 29960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:16.334900 29966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:16.334924 29967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:16.336668 29969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:16.337167 29960 server_base.cc:1034] running on GCE node
I20260808 15:07:16.337672 29960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:16.338483 29960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:16.339679 29960 hybrid_clock.cc:648] HybridClock initialized: now 1786201636339623 us; error 48 us; skew 500 ppm
I20260808 15:07:16.339998 29960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:16 dist-test-slave-jlzn krb5kdc[29804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:16.342904 29960 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:07:16.344323 29960 webserver.cc:460] Webserver started at http://127.2.180.193:44327/ using document root <none> and password file <none>
I20260808 15:07:16.344913 29960 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:16.345052 29960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:16.345350 29960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:16.348066 29960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "bb2302e6332f4e80a20b6d5a753c73cc"
format_stamp: "Formatted at 2026-08-08 15:07:16 on dist-test-slave-jlzn"
server_key: "9c9257ba2ed835afa29eff0962c12b34"
server_key_iv: "c561813ec7fa3115e4cc0346085e9d8e"
server_key_version: "encryptionkey@0"
I20260808 15:07:16.348809 29960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "bb2302e6332f4e80a20b6d5a753c73cc"
format_stamp: "Formatted at 2026-08-08 15:07:16 on dist-test-slave-jlzn"
server_key: "9c9257ba2ed835afa29eff0962c12b34"
server_key_iv: "c561813ec7fa3115e4cc0346085e9d8e"
server_key_version: "encryptionkey@0"
I20260808 15:07:16.353497 29960 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:07:16.356683 29980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:16.357820 29960 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:07:16.357959 29960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "bb2302e6332f4e80a20b6d5a753c73cc"
format_stamp: "Formatted at 2026-08-08 15:07:16 on dist-test-slave-jlzn"
server_key: "9c9257ba2ed835afa29eff0962c12b34"
server_key_iv: "c561813ec7fa3115e4cc0346085e9d8e"
server_key_version: "encryptionkey@0"
I20260808 15:07:16.358134 29960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:16.373842 29960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:16.377264 29960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:16.377513 29960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:16.378377 29960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:16.379614 29960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:16.379691 29960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:16.379839 29960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:16.379900 29960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 15:07:16.409755 29960 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:38167
I20260808 15:07:16.409801 30159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:38167 every 8 connection(s)
I20260808 15:07:16.410921 29960 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:07:16.414605 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 29960
I20260808 15:07:16.414790 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:07:16.415194 2771 external_mini_cluster.cc:1428] Setting key b6b87d9004f21f8588b4d52348eb011e
Aug 08 15:07:16 dist-test-slave-jlzn krb5kdc[29804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:16.429706 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:16.413235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:36617 (local address 127.2.180.254:35465)
0808 15:07:16.413677 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:16.413684 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:16.414606 (+ 922us) server_negotiation.cc:407] Connection header received
0808 15:07:16.416589 (+ 1983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:16.416594 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:16.416671 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:16.416763 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:16.418198 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:16.418922 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:16.420479 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:16.420880 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:16.420996 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:16.424412 (+ 3416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:16.424417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:16.424420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:16.424451 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:16.426853 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:16.427524 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:16.427529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:16.427531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:16.427652 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:16.428103 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:16.428106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:16.428108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:16.428358 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:16.428490 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:16.429139 (+ 649us) server_negotiation.cc:299] Negotiation successful
0808 15:07:16.429355 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":196,"threads_started":1}
I20260808 15:07:16.430552 30165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:16.413768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35465 (local address 127.2.180.193:36617)
0808 15:07:16.414457 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:16.414493 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:16.416327 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:16.416995 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:16.417003 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:16.417491 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:16.418040 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:16.418057 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:16.419084 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:16.419088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:16.420344 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:16.420356 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:16.421117 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:16.421124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:16.421347 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:16.421962 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:16.421990 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:16.424215 (+ 2225us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:16.426979 (+ 2764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:16.426988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:16.427002 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:16.427397 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:16.427757 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:16.427762 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:16.427765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:16.427982 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:16.428483 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:16.428490 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:16.428787 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 15:07:16.429006 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 15:07:16.429477 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":282,"threads_started":1}
I20260808 15:07:16.432039 30160 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35465
I20260808 15:07:16.432300 30160 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:16.432967 30160 heartbeater.cc:511] Master 127.2.180.254:35465 requested a full tablet report, sending...
I20260808 15:07:16.435137 29871 ts_manager.cc:194] Registered new tserver with Master: bb2302e6332f4e80a20b6d5a753c73cc (127.2.180.193:38167)
I20260808 15:07:16.437054 29871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:36617
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:07:16.472678 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35465
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:16.788642 30173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:16.788885 30173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:16.788985 30173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:16.796002 30173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:16.796099 30173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:16.796209 30173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:07:16.805317 30173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:16.806428 30173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:16.807590 30173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:16.819751 30180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:16.819774 30182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:16.819720 30185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:16.820840 30173 server_base.cc:1034] running on GCE node
I20260808 15:07:16.821394 30173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:16.822208 30173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:16.823431 30173 hybrid_clock.cc:648] HybridClock initialized: now 1786201636823379 us; error 36 us; skew 500 ppm
I20260808 15:07:16.823788 30173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:16 dist-test-slave-jlzn krb5kdc[29804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:16.827534 30173 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:07:16.829144 30173 webserver.cc:460] Webserver started at http://127.2.180.194:43173/ using document root <none> and password file <none>
I20260808 15:07:16.829924 30173 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:16.830044 30173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:16.830361 30173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:16.833900 30173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "8e9a68ce4b8940a9affccbc20c13f248"
format_stamp: "Formatted at 2026-08-08 15:07:16 on dist-test-slave-jlzn"
server_key: "20f56715a3e79c6e3eb21d559b5bd862"
server_key_iv: "624815fe1dbbd9f50bca2b9d836b0c52"
server_key_version: "encryptionkey@0"
I20260808 15:07:16.834805 30173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "8e9a68ce4b8940a9affccbc20c13f248"
format_stamp: "Formatted at 2026-08-08 15:07:16 on dist-test-slave-jlzn"
server_key: "20f56715a3e79c6e3eb21d559b5bd862"
server_key_iv: "624815fe1dbbd9f50bca2b9d836b0c52"
server_key_version: "encryptionkey@0"
I20260808 15:07:16.840916 30173 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260808 15:07:16.844761 30194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:16.845911 30173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260808 15:07:16.846041 30173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "8e9a68ce4b8940a9affccbc20c13f248"
format_stamp: "Formatted at 2026-08-08 15:07:16 on dist-test-slave-jlzn"
server_key: "20f56715a3e79c6e3eb21d559b5bd862"
server_key_iv: "624815fe1dbbd9f50bca2b9d836b0c52"
server_key_version: "encryptionkey@0"
I20260808 15:07:16.846205 30173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:16.869325 30173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:16.873345 30173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:16.873595 30173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:16.874447 30173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:16.875627 30173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:16.875705 30173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:16.875841 30173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:16.875909 30173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 15:07:16.904829 30173 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:45521
I20260808 15:07:16.904874 30365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:45521 every 8 connection(s)
I20260808 15:07:16.906003 30173 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:07:16.913903 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 30173
I20260808 15:07:16.914059 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:07:16.914466 2771 external_mini_cluster.cc:1428] Setting key 0adf4d3f89cdb6441498377fb171f248
Aug 08 15:07:16 dist-test-slave-jlzn krb5kdc[29804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201636, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:16.924301 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:16.908238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:58773 (local address 127.2.180.254:35465)
0808 15:07:16.908407 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:16.908411 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:16.909475 (+ 1064us) server_negotiation.cc:407] Connection header received
0808 15:07:16.911254 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:16.911260 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:16.911341 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:16.911428 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:16.912882 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:16.913570 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:16.915128 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:16.915601 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:16.915722 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:16.919002 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:16.919007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:16.919010 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:16.919041 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:16.921473 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:16.922133 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:16.922137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:16.922139 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:16.922268 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:16.922670 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:16.922674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:16.922676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:16.922934 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:16.923066 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:16.923701 (+ 635us) server_negotiation.cc:299] Negotiation successful
0808 15:07:16.923954 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:07:16.925107 30368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:16.908672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35465 (local address 127.2.180.194:58773)
0808 15:07:16.909319 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:16.909364 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:16.911002 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:16.911682 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:16.911690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:16.912196 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:16.912719 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:16.912736 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:16.913720 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:16.913725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:16.914981 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:16.914994 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:16.915833 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:16.915841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:16.916049 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:16.916626 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:16.916648 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:16.918799 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:16.921608 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:16.921617 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:16.921631 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:16.922018 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:16.922372 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:16.922377 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:16.922381 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:16.922555 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:16.923064 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:16.923072 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:16.923362 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 15:07:16.923594 (+ 232us) client_negotiation.cc:240] Negotiation successful
0808 15:07:16.924086 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":252,"threads_started":1}
I20260808 15:07:16.926355 30366 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35465
I20260808 15:07:16.926651 30366 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:16.927423 30366 heartbeater.cc:511] Master 127.2.180.254:35465 requested a full tablet report, sending...
I20260808 15:07:16.928869 29871 ts_manager.cc:194] Registered new tserver with Master: 8e9a68ce4b8940a9affccbc20c13f248 (127.2.180.194:45521)
I20260808 15:07:16.929607 29871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:58773
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:07:16.970580 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35465
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:17.279013 30375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:17.279285 30375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:17.279428 30375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:17.286160 30375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:17.286252 30375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:17.286388 30375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:07:17.295336 30375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:35465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:17.296483 30375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:17.297721 30375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:17.309928 30381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:17.310169 30380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:17.310478 30383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:17.310577 30375 server_base.cc:1034] running on GCE node
I20260808 15:07:17.311179 30375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:17.312111 30375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:17.313299 30375 hybrid_clock.cc:648] HybridClock initialized: now 1786201637313259 us; error 33 us; skew 500 ppm
I20260808 15:07:17.313573 30375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:17 dist-test-slave-jlzn krb5kdc[29804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201637, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:17.316486 30375 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:07:17.317726 30375 webserver.cc:460] Webserver started at http://127.2.180.195:39875/ using document root <none> and password file <none>
I20260808 15:07:17.318290 30375 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:17.318387 30375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:17.318634 30375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:17.321376 30375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "f855e7494eed440181b6273fadc5f319"
format_stamp: "Formatted at 2026-08-08 15:07:17 on dist-test-slave-jlzn"
server_key: "90119bc94f91cd611f64a554a46c4120"
server_key_iv: "b711913b632864e70ec122112b2ad0e8"
server_key_version: "encryptionkey@0"
I20260808 15:07:17.322088 30375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "f855e7494eed440181b6273fadc5f319"
format_stamp: "Formatted at 2026-08-08 15:07:17 on dist-test-slave-jlzn"
server_key: "90119bc94f91cd611f64a554a46c4120"
server_key_iv: "b711913b632864e70ec122112b2ad0e8"
server_key_version: "encryptionkey@0"
I20260808 15:07:17.326846 30375 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:07:17.330123 30393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:17.331391 30375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 15:07:17.331525 30375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "f855e7494eed440181b6273fadc5f319"
format_stamp: "Formatted at 2026-08-08 15:07:17 on dist-test-slave-jlzn"
server_key: "90119bc94f91cd611f64a554a46c4120"
server_key_iv: "b711913b632864e70ec122112b2ad0e8"
server_key_version: "encryptionkey@0"
I20260808 15:07:17.331682 30375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:17.350449 30375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:17.353871 30375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:17.354099 30375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:17.354879 30375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:17.356091 30375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:17.356164 30375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:17.356256 30375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:17.356297 30375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:17.385294 30375 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:45123
I20260808 15:07:17.385353 30560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:45123 every 8 connection(s)
I20260808 15:07:17.386416 30375 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:07:17.390525 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 30375
I20260808 15:07:17.390686 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:07:17.391084 2771 external_mini_cluster.cc:1428] Setting key ba3bb1e365bbe74b354e8f7e8e466b0a
Aug 08 15:07:17 dist-test-slave-jlzn krb5kdc[29804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201637, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:17.404855 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.388627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:51331 (local address 127.2.180.254:35465)
0808 15:07:17.388832 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.388839 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.389915 (+ 1076us) server_negotiation.cc:407] Connection header received
0808 15:07:17.391824 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.391831 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.391911 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.392005 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:17.393817 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.394576 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.396048 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.396428 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.396518 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.399495 (+ 2977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:17.399499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:17.399502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:17.399534 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:17.402030 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:17.402673 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:17.402679 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:17.402681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:17.402811 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:17.403234 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:17.403239 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:17.403241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:17.403500 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:17.403641 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.404298 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.404519 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:07:17.405798 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.389115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:35465 (local address 127.2.180.195:51331)
0808 15:07:17.389765 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:17.389810 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:17.391442 (+ 1632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:17.392282 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:17.392293 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:17.392897 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:17.393652 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.393679 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.394710 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.394714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.395945 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.395957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.396589 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.396593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.396766 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.397290 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:17.397314 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:17.399310 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:17.402159 (+ 2849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:17.402167 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:17.402181 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:17.402560 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:17.402915 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:17.402920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:17.402925 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:17.403128 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:17.403624 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:17.403632 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:17.403949 (+ 317us) client_negotiation.cc:769] Sending connection context
0808 15:07:17.404186 (+ 237us) client_negotiation.cc:240] Negotiation successful
0808 15:07:17.404626 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":252,"threads_started":1}
I20260808 15:07:17.406914 30561 heartbeater.cc:347] Connected to a master server at 127.2.180.254:35465
I20260808 15:07:17.407155 30561 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:17.407831 30561 heartbeater.cc:511] Master 127.2.180.254:35465 requested a full tablet report, sending...
I20260808 15:07:17.409107 29871 ts_manager.cc:194] Registered new tserver with Master: f855e7494eed440181b6273fadc5f319 (127.2.180.195:45123)
I20260808 15:07:17.409897 29871 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:51331
I20260808 15:07:17.417505 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:07:17.433362 30162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.421387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.2.180.254:35465)
0808 15:07:17.421537 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.421541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.421958 (+ 417us) server_negotiation.cc:407] Connection header received
0808 15:07:17.422096 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.422100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.422154 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.422220 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:17.423522 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.424261 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.425524 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.426010 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.426114 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.428664 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:17.428669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:17.428673 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:17.428703 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:17.431133 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:17.431777 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:17.431780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:17.431782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:17.431939 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:17.432278 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:17.432282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:17.432283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:17.432578 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:17.432764 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.433015 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.433142 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:07:17.439736 30160 heartbeater.cc:503] Master 127.2.180.254:35465 was elected leader, sending a full tablet report...
I20260808 15:07:17.436910 29871 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:07:17.443764 29871 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:07:17.477432 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.454736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38167 (local address 127.0.0.1:38004)
0808 15:07:17.455831 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:17.455881 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:17.456042 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:17.470368 (+ 14326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:17.470377 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:17.470413 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:17.470927 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.470948 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.472611 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.472616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.474833 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.474845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.476396 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.476402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.476602 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.476651 (+ 49us) client_negotiation.cc:769] Sending connection context
0808 15:07:17.476779 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 15:07:17.476946 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"mutex_wait_us":27,"thread_start_us":715,"threads_started":1}
I20260808 15:07:17.478379 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.454866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.2.180.193:38167)
0808 15:07:17.469760 (+ 14894us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.469768 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.469797 (+ 29us) server_negotiation.cc:407] Connection header received
0808 15:07:17.469890 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.469897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.470136 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.470326 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:17.471134 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.472487 (+ 1353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.474982 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.476217 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.476354 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.476575 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.477916 (+ 1341us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.478075 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14723,"thread_start_us":14577,"threads_started":1}
I20260808 15:07:17.478440 30583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.455447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:45521 (local address 127.0.0.1:55914)
0808 15:07:17.469392 (+ 13945us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:17.469409 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:17.469523 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:17.471101 (+ 1578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:17.471105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:17.471155 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:17.471650 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.471664 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.473529 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.473534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.475316 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.475332 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.477352 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.477359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.477796 (+ 437us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.477832 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 15:07:17.477896 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 15:07:17.478015 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13811,"mutex_wait_us":25,"spinlock_wait_cycles":3840,"thread_start_us":15166,"threads_started":1}
I20260808 15:07:17.478799 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.456587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36480 (local address 127.2.180.195:45123)
0808 15:07:17.468814 (+ 12227us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.468824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.471036 (+ 2212us) server_negotiation.cc:407] Connection header received
0808 15:07:17.471328 (+ 292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.471336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.471584 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.471743 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:17.472716 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.474285 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.476278 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.477672 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.477790 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.478022 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.478243 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.478403 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12050,"thread_start_us":12231,"threads_started":1}
I20260808 15:07:17.479023 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.455670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55914 (local address 127.2.180.194:45521)
0808 15:07:17.470017 (+ 14347us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.470025 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.470049 (+ 24us) server_negotiation.cc:407] Connection header received
0808 15:07:17.470136 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.470144 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.470351 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.471845 (+ 1494us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:17.472085 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.473398 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.475490 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.477066 (+ 1576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.478137 (+ 1071us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.478388 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.478462 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.478623 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14194,"thread_start_us":14065,"threads_started":1}
I20260808 15:07:17.479377 30576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.456253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:45123 (local address 127.0.0.1:36480)
0808 15:07:17.470909 (+ 14656us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:17.470929 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:17.471139 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:17.472108 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:17.472112 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:17.472137 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:17.472467 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.472479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.474428 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.474434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.476135 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.476148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.477886 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.477893 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.478045 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.478073 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 15:07:17.478969 (+ 896us) client_negotiation.cc:240] Negotiation successful
0808 15:07:17.479095 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14526,"mutex_wait_us":1,"thread_start_us":12874,"threads_started":1}
I20260808 15:07:17.482290 30274 tablet_service.cc:1467] Processing CreateTablet for tablet 3603f1f176834d3bbd68e748be69bc0b (DEFAULT_TABLE table=test-table [id=c81719826f434240b3ee072db574fdf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:17.482262 30060 tablet_service.cc:1467] Processing CreateTablet for tablet 3603f1f176834d3bbd68e748be69bc0b (DEFAULT_TABLE table=test-table [id=c81719826f434240b3ee072db574fdf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:17.482589 30467 tablet_service.cc:1467] Processing CreateTablet for tablet 3603f1f176834d3bbd68e748be69bc0b (DEFAULT_TABLE table=test-table [id=c81719826f434240b3ee072db574fdf0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:17.483489 30274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3603f1f176834d3bbd68e748be69bc0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:17.483628 30060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3603f1f176834d3bbd68e748be69bc0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:17.484078 30467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3603f1f176834d3bbd68e748be69bc0b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:17.494141 30586 tablet_bootstrap.cc:492] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Bootstrap starting.
I20260808 15:07:17.497300 30587 tablet_bootstrap.cc:492] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Bootstrap starting.
I20260808 15:07:17.497397 30586 tablet_bootstrap.cc:654] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:17.498080 30589 tablet_bootstrap.cc:492] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Bootstrap starting.
I20260808 15:07:17.499042 30586 log.cc:824] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:17.501434 30589 tablet_bootstrap.cc:654] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:17.501495 30586 tablet_bootstrap.cc:492] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: No bootstrap required, opened a new log
I20260808 15:07:17.501726 30586 ts_tablet_manager.cc:1397] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260808 15:07:17.502638 30589 log.cc:824] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:17.503072 30587 tablet_bootstrap.cc:654] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:17.504452 30587 log.cc:824] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:17.507138 30587 tablet_bootstrap.cc:492] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: No bootstrap required, opened a new log
I20260808 15:07:17.507367 30587 ts_tablet_manager.cc:1397] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 15:07:17.507524 30589 tablet_bootstrap.cc:492] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: No bootstrap required, opened a new log
I20260808 15:07:17.507781 30589 ts_tablet_manager.cc:1397] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260808 15:07:17.510862 30586 raft_consensus.cc:348] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.511269 30586 raft_consensus.cc:374] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:17.511369 30586 raft_consensus.cc:729] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e9a68ce4b8940a9affccbc20c13f248, State: Initialized, Role: FOLLOWER
I20260808 15:07:17.511984 30586 consensus_queue.cc:260] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.513038 30366 heartbeater.cc:503] Master 127.2.180.254:35465 was elected leader, sending a full tablet report...
I20260808 15:07:17.513613 30586 ts_tablet_manager.cc:1428] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Time spent starting tablet: real 0.012s user 0.003s sys 0.009s
I20260808 15:07:17.515033 30587 raft_consensus.cc:348] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.515337 30587 raft_consensus.cc:374] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:17.515430 30587 raft_consensus.cc:729] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb2302e6332f4e80a20b6d5a753c73cc, State: Initialized, Role: FOLLOWER
I20260808 15:07:17.516029 30587 consensus_queue.cc:260] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.517096 30589 raft_consensus.cc:348] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.517413 30589 raft_consensus.cc:374] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:17.517504 30589 raft_consensus.cc:729] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f855e7494eed440181b6273fadc5f319, State: Initialized, Role: FOLLOWER
I20260808 15:07:17.518019 30589 consensus_queue.cc:260] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.519098 30561 heartbeater.cc:503] Master 127.2.180.254:35465 was elected leader, sending a full tablet report...
I20260808 15:07:17.519630 30589 ts_tablet_manager.cc:1428] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Time spent starting tablet: real 0.012s user 0.004s sys 0.007s
I20260808 15:07:17.521202 30587 ts_tablet_manager.cc:1428] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260808 15:07:17.586899 30595 raft_consensus.cc:482] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:07:17.587185 30595 raft_consensus.cc:504] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.588876 30595 leader_election.cc:288] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bb2302e6332f4e80a20b6d5a753c73cc (127.2.180.193:38167), f855e7494eed440181b6273fadc5f319 (127.2.180.195:45123)
I20260808 15:07:17.594941 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.589194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:47789 (local address 127.2.180.193:38167)
0808 15:07:17.589374 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.589378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.589826 (+ 448us) server_negotiation.cc:407] Connection header received
0808 15:07:17.590045 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.590051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.590132 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.590238 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:17.590805 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.591521 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.593295 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.594076 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.594229 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.594306 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.594586 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.594711 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:07:17.595014 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.589131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:38167 (local address 127.2.180.194:47789)
0808 15:07:17.589730 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:17.589755 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:17.589889 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:17.590282 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:17.590287 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:17.590315 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:17.590661 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.590671 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.591658 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.591662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.593139 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.593152 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.594290 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.594296 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.594431 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.594454 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:07:17.594538 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 15:07:17.594662 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":286,"threads_started":1}
I20260808 15:07:17.595337 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.590086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:58557 (local address 127.2.180.195:45123)
0808 15:07:17.590287 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.590292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.590534 (+ 242us) server_negotiation.cc:407] Connection header received
0808 15:07:17.590726 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.590729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.590786 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.590853 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:17.591370 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.592137 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.593745 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.594478 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.594623 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.594703 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.594986 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.595103 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:07:17.595453 30599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.589851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:45123 (local address 127.2.180.194:58557)
0808 15:07:17.590430 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:17.590446 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:17.590578 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:17.590884 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:17.590889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:17.590911 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:17.591216 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.591225 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.592273 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.592277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.593597 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:17.593609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.594686 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.594693 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:17.594849 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.594872 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:07:17.594947 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 15:07:17.595069 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"spinlock_wait_cycles":4864,"thread_start_us":436,"threads_started":1}
I20260808 15:07:17.596117 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3603f1f176834d3bbd68e748be69bc0b" candidate_uuid: "8e9a68ce4b8940a9affccbc20c13f248" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" is_pre_election: true
I20260808 15:07:17.596472 30494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3603f1f176834d3bbd68e748be69bc0b" candidate_uuid: "8e9a68ce4b8940a9affccbc20c13f248" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f855e7494eed440181b6273fadc5f319" is_pre_election: true
I20260808 15:07:17.596716 30091 raft_consensus.cc:2454] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9a68ce4b8940a9affccbc20c13f248 in term 0.
I20260808 15:07:17.597024 30494 raft_consensus.cc:2454] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e9a68ce4b8940a9affccbc20c13f248 in term 0.
I20260808 15:07:17.597397 30195 leader_election.cc:302] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9a68ce4b8940a9affccbc20c13f248, bb2302e6332f4e80a20b6d5a753c73cc; no voters:
I20260808 15:07:17.597755 30595 raft_consensus.cc:2781] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:07:17.597889 30595 raft_consensus.cc:482] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:07:17.597980 30595 raft_consensus.cc:3037] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:17.600183 30595 raft_consensus.cc:504] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.600719 30595 leader_election.cc:288] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [CANDIDATE]: Term 1 election: Requested vote from peers bb2302e6332f4e80a20b6d5a753c73cc (127.2.180.193:38167), f855e7494eed440181b6273fadc5f319 (127.2.180.195:45123)
I20260808 15:07:17.601133 30091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3603f1f176834d3bbd68e748be69bc0b" candidate_uuid: "8e9a68ce4b8940a9affccbc20c13f248" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb2302e6332f4e80a20b6d5a753c73cc"
I20260808 15:07:17.601184 30494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3603f1f176834d3bbd68e748be69bc0b" candidate_uuid: "8e9a68ce4b8940a9affccbc20c13f248" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f855e7494eed440181b6273fadc5f319"
I20260808 15:07:17.601339 30091 raft_consensus.cc:3037] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:17.601368 30494 raft_consensus.cc:3037] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:17.603456 30091 raft_consensus.cc:2454] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9a68ce4b8940a9affccbc20c13f248 in term 1.
I20260808 15:07:17.603462 30494 raft_consensus.cc:2454] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e9a68ce4b8940a9affccbc20c13f248 in term 1.
I20260808 15:07:17.603789 30195 leader_election.cc:302] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e9a68ce4b8940a9affccbc20c13f248, f855e7494eed440181b6273fadc5f319; no voters:
I20260808 15:07:17.604080 30595 raft_consensus.cc:2781] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:17.604502 30595 raft_consensus.cc:686] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 1 LEADER]: Becoming Leader. State: Replica: 8e9a68ce4b8940a9affccbc20c13f248, State: Running, Role: LEADER
I20260808 15:07:17.605041 30595 consensus_queue.cc:237] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } }
I20260808 15:07:17.609575 29869 catalog_manager.cc:5495] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e9a68ce4b8940a9affccbc20c13f248 (127.2.180.194). New cstate: current_term: 1 leader_uuid: "8e9a68ce4b8940a9affccbc20c13f248" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e9a68ce4b8940a9affccbc20c13f248" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 45521 } health_report { overall_health: HEALTHY } } }
W20260808 15:07:17.637852 30562 tablet.cc:2369] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:07:17.652863 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.647961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55916 (local address 127.2.180.194:45521)
0808 15:07:17.648133 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.648137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.648252 (+ 115us) server_negotiation.cc:407] Connection header received
0808 15:07:17.648404 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.648408 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.648488 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.648609 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:07:17.648951 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.649696 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.651045 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.651446 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.651553 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.651994 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:07:17.652433 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:07:17.652574 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.652626 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.652663 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 15:07:17.657785 30367 tablet.cc:2369] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:07:17.660650 30494 raft_consensus.cc:1261] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 1 FOLLOWER]: Refusing update from remote peer 8e9a68ce4b8940a9affccbc20c13f248: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:17.660657 30091 raft_consensus.cc:1261] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 1 FOLLOWER]: Refusing update from remote peer 8e9a68ce4b8940a9affccbc20c13f248: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:17.661471 30601 consensus_queue.cc:1035] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb2302e6332f4e80a20b6d5a753c73cc" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 38167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:17.661818 30595 consensus_queue.cc:1035] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f855e7494eed440181b6273fadc5f319" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 45123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 15:07:17.662988 30161 tablet.cc:2369] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:07:17.687860 29871 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:17.688133 29871 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:17.691529 29871 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b2555ec1d164bd484f4b3e97ac67fb7: Sending DeleteTablet for 3 replicas of tablet 3603f1f176834d3bbd68e748be69bc0b
I20260808 15:07:17.692512 30060 tablet_service.cc:1514] Processing DeleteTablet for tablet 3603f1f176834d3bbd68e748be69bc0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:17 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:38004
I20260808 15:07:17.692566 30274 tablet_service.cc:1514] Processing DeleteTablet for tablet 3603f1f176834d3bbd68e748be69bc0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:17 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:55914
I20260808 15:07:17.693145 30467 tablet_service.cc:1514] Processing DeleteTablet for tablet 3603f1f176834d3bbd68e748be69bc0b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:17 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:36480
I20260808 15:07:17.693351 30619 tablet_replica.cc:330] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: stopping tablet replica
I20260808 15:07:17.693737 30619 raft_consensus.cc:2229] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:07:17.694473 30619 raft_consensus.cc:2258] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:17.695431 30623 tablet_replica.cc:330] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: stopping tablet replica
I20260808 15:07:17.695834 30623 raft_consensus.cc:2229] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:17.696288 30623 raft_consensus.cc:2258] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:17.698052 30623 ts_tablet_manager.cc:1905] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:17.698055 30619 ts_tablet_manager.cc:1905] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:17.698156 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 29960
I20260808 15:07:17.704077 30620 tablet_replica.cc:330] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: stopping tablet replica
I20260808 15:07:17.704470 30620 raft_consensus.cc:2229] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:17.704902 30620 raft_consensus.cc:2258] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:17.705157 30619 ts_tablet_manager.cc:1918] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:17.705317 30619 log.cc:1196] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/3603f1f176834d3bbd68e748be69bc0b
I20260808 15:07:17.705967 30619 ts_tablet_manager.cc:1939] T 3603f1f176834d3bbd68e748be69bc0b P 8e9a68ce4b8940a9affccbc20c13f248: Deleting consensus metadata
I20260808 15:07:17.706161 30623 ts_tablet_manager.cc:1918] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:17.706387 30623 log.cc:1196] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/3603f1f176834d3bbd68e748be69bc0b
I20260808 15:07:17.707131 30623 ts_tablet_manager.cc:1939] T 3603f1f176834d3bbd68e748be69bc0b P f855e7494eed440181b6273fadc5f319: Deleting consensus metadata
I20260808 15:07:17.707659 29844 catalog_manager.cc:4841] TS 8e9a68ce4b8940a9affccbc20c13f248 (127.2.180.194:45521): tablet 3603f1f176834d3bbd68e748be69bc0b (table test-table [id=c81719826f434240b3ee072db574fdf0]) successfully deleted
I20260808 15:07:17.708701 30620 ts_tablet_manager.cc:1905] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:17.709197 29843 catalog_manager.cc:4841] TS f855e7494eed440181b6273fadc5f319 (127.2.180.195:45123): tablet 3603f1f176834d3bbd68e748be69bc0b (table test-table [id=c81719826f434240b3ee072db574fdf0]) successfully deleted
I20260808 15:07:17.714802 30620 ts_tablet_manager.cc:1918] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:17.714982 30620 log.cc:1196] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/3603f1f176834d3bbd68e748be69bc0b
I20260808 15:07:17.715442 30620 ts_tablet_manager.cc:1939] T 3603f1f176834d3bbd68e748be69bc0b P bb2302e6332f4e80a20b6d5a753c73cc: Deleting consensus metadata
I20260808 15:07:17.716903 29843 catalog_manager.cc:4841] TS bb2302e6332f4e80a20b6d5a753c73cc (127.2.180.193:38167): tablet 3603f1f176834d3bbd68e748be69bc0b (table test-table [id=c81719826f434240b3ee072db574fdf0]) successfully deleted
Aug 08 15:07:17 dist-test-slave-jlzn krb5kdc[29804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:07:17.726302 30575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.701180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38006 (local address 127.2.180.193:38167)
0808 15:07:17.701342 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.701347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.709944 (+ 8597us) server_negotiation.cc:407] Connection header received
0808 15:07:17.710108 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.710114 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.710219 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.710339 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:17.711925 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.713108 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.714889 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.715582 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.715712 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.720017 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:17.720031 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:17.720048 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:17.720088 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:17.723588 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:17.724289 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:17.724298 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:17.724301 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:17.724455 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:17.724784 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:17.724789 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:17.724792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:17.725248 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:17.725554 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.725728 (+ 174us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.725944 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 15:07:17.726929 30155 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:17.820953 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 29960
I20260808 15:07:17.847445 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 30173
Aug 08 15:07:17 dist-test-slave-jlzn krb5kdc[29804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:07:17.864324 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.848224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55930 (local address 127.2.180.194:45521)
0808 15:07:17.848399 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.848403 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.848824 (+ 421us) server_negotiation.cc:407] Connection header received
0808 15:07:17.848998 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.849002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.849067 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.849150 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:17.850560 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.851429 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.852806 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.853251 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:17.853364 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:17.857663 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:17.857678 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:17.857699 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:17.857747 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:17.861461 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:17.862081 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:17.862091 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:17.862094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:17.862275 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:17.862692 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:17.862698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:17.862701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:17.863056 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:17.863423 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:17.863680 (+ 257us) server_negotiation.cc:299] Negotiation successful
0808 15:07:17.864004 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 15:07:17.865046 30360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:17.972591 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 30173
I20260808 15:07:17.995723 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 30375
Aug 08 15:07:18 dist-test-slave-jlzn krb5kdc[29804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:07:18.009820 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:17.996464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36496 (local address 127.2.180.195:45123)
0808 15:07:17.996651 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:17.996657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:17.996710 (+ 53us) server_negotiation.cc:407] Connection header received
0808 15:07:17.996908 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:17.996912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:17.996978 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:17.997060 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:17.998357 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:17.999081 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:18.000407 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:18.000807 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:18.000915 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:18.004974 (+ 4059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:18.004986 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:18.005002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:18.005039 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:18.007416 (+ 2377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:18.008205 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:18.008212 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:18.008214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:18.008356 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:18.008642 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:18.008645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:18.008647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:18.008981 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:18.009200 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:18.009403 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 15:07:18.009586 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:07:18.010427 30554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:18.094930 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 30375
I20260808 15:07:18.114660 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 29822
I20260808 15:07:18.115783 29921 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:18.220952 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 29822
2026-08-08T15:07:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4237 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T15:07:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:18Z Disabled control of system clock
I20260808 15:07:18.301203 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46301
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:33671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:46301
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:07:18.618955 30635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:18.619216 30635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:18.619302 30635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:18.626312 30635 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:18.626411 30635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:18.626456 30635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:18.626519 30635 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:18.626565 30635 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:18.635290 30635 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:46301
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46301
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:18.636441 30635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:18.638105 30635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:18.648965 30641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:18.649037 30640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:18.649004 30643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:18.649868 30635 server_base.cc:1034] running on GCE node
I20260808 15:07:18.650558 30635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:18.651844 30635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:18.653047 30635 hybrid_clock.cc:648] HybridClock initialized: now 1786201638652997 us; error 45 us; skew 500 ppm
I20260808 15:07:18.653357 30635 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:18.655418 30635 webserver.cc:460] Webserver started at http://127.2.180.254:36217/ using document root <none> and password file <none>
I20260808 15:07:18.656064 30635 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:18.656170 30635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:18.656455 30635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:18.659076 30635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "edc7f6728517427db8c44d7980fd6178"
format_stamp: "Formatted at 2026-08-08 15:07:18 on dist-test-slave-jlzn"
server_key: "23a5bb68358f7ec44ec5b515a7829f2b"
server_key_iv: "16d62fe230ef3ff988dd491ad966ed17"
server_key_version: "encryptionkey@0"
I20260808 15:07:18.659874 30635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "edc7f6728517427db8c44d7980fd6178"
format_stamp: "Formatted at 2026-08-08 15:07:18 on dist-test-slave-jlzn"
server_key: "23a5bb68358f7ec44ec5b515a7829f2b"
server_key_iv: "16d62fe230ef3ff988dd491ad966ed17"
server_key_version: "encryptionkey@0"
I20260808 15:07:18.664590 30635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 15:07:18.667675 30655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:18.668870 30635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:07:18.669034 30635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "edc7f6728517427db8c44d7980fd6178"
format_stamp: "Formatted at 2026-08-08 15:07:18 on dist-test-slave-jlzn"
server_key: "23a5bb68358f7ec44ec5b515a7829f2b"
server_key_iv: "16d62fe230ef3ff988dd491ad966ed17"
server_key_version: "encryptionkey@0"
I20260808 15:07:18.669220 30635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:18.684537 30635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:18.685209 30635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:18.685449 30635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:18.701963 30635 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:46301
I20260808 15:07:18.701982 30725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:46301 every 8 connection(s)
I20260808 15:07:18.703087 30635 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:18.706401 30726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:18.710647 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 30635
I20260808 15:07:18.710832 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:18.711356 2771 external_mini_cluster.cc:1428] Setting key 098f91421fa554ee64ef9f3f8da8b501
I20260808 15:07:18.717409 30726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178: Bootstrap starting.
I20260808 15:07:18.720695 30726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:18.721853 30726 log.cc:824] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:18.724696 30726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178: No bootstrap required, opened a new log
I20260808 15:07:18.724713 30730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:18.713183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55006 (local address 127.2.180.254:46301)
0808 15:07:18.714076 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:18.714088 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:18.714133 (+ 45us) server_negotiation.cc:407] Connection header received
0808 15:07:18.716075 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:18.716096 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:18.716475 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:18.716773 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:18.717331 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:18.718596 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:18.720451 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:18.721151 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:18.721273 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:18.722496 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:18.722507 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:18.722524 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:18.723177 (+ 653us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:18.723258 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:18.723278 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:18.723359 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:18.723572 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 15:07:18.723864 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":341,"threads_started":1}
I20260808 15:07:18.732033 30726 raft_consensus.cc:348] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc7f6728517427db8c44d7980fd6178" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46301 } }
I20260808 15:07:18.732348 30726 raft_consensus.cc:374] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:18.732445 30726 raft_consensus.cc:729] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edc7f6728517427db8c44d7980fd6178, State: Initialized, Role: FOLLOWER
I20260808 15:07:18.732930 30726 consensus_queue.cc:260] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc7f6728517427db8c44d7980fd6178" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46301 } }
I20260808 15:07:18.733155 30726 raft_consensus.cc:388] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:18.733282 30726 raft_consensus.cc:482] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:18.733438 30726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:18.735464 30726 raft_consensus.cc:504] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc7f6728517427db8c44d7980fd6178" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46301 } }
I20260808 15:07:18.736002 30726 leader_election.cc:302] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edc7f6728517427db8c44d7980fd6178; no voters:
I20260808 15:07:18.736583 30726 leader_election.cc:288] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:18.736702 30732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:18.737376 30732 raft_consensus.cc:686] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [term 1 LEADER]: Becoming Leader. State: Replica: edc7f6728517427db8c44d7980fd6178, State: Running, Role: LEADER
I20260808 15:07:18.737941 30732 consensus_queue.cc:237] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc7f6728517427db8c44d7980fd6178" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46301 } }
I20260808 15:07:18.738642 30726 sys_catalog.cc:564] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:18.740600 30733 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edc7f6728517427db8c44d7980fd6178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc7f6728517427db8c44d7980fd6178" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46301 } } }
I20260808 15:07:18.741004 30733 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:18.742740 30740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:18.742585 30734 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edc7f6728517427db8c44d7980fd6178. Latest consensus state: current_term: 1 leader_uuid: "edc7f6728517427db8c44d7980fd6178" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc7f6728517427db8c44d7980fd6178" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46301 } } }
I20260808 15:07:18.743283 30734 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:18.748370 30740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:18.756716 30740 catalog_manager.cc:1347] Generated new cluster ID: b3e11c62a6a24a67b9ecb48933736b44
I20260808 15:07:18.756809 30740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:18.767712 30740 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:18.768939 30740 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:18.775498 30740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178: Generated new TSK 0
I20260808 15:07:18.776405 30740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:18.791440 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46301
--builtin_ntp_servers=127.2.180.212:33671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:07:19.116812 30756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:19.117084 30756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:19.117211 30756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:19.124141 30756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:19.124243 30756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:19.124382 30756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:07:19.132982 30756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:19.134119 30756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:19.135846 30756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:19.148001 30763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:19.148052 30764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:19.147967 30766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:19.149024 30756 server_base.cc:1034] running on GCE node
I20260808 15:07:19.149571 30756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:19.150406 30756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:19.151623 30756 hybrid_clock.cc:648] HybridClock initialized: now 1786201639151573 us; error 37 us; skew 500 ppm
I20260808 15:07:19.151953 30756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:19.158478 30756 webserver.cc:460] Webserver started at http://127.2.180.193:39833/ using document root <none> and password file <none>
I20260808 15:07:19.159170 30756 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:19.159276 30756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:19.159574 30756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:19.162338 30756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "e563a0c5c36a44efad60dcefafd511d2"
format_stamp: "Formatted at 2026-08-08 15:07:19 on dist-test-slave-jlzn"
server_key: "69b12100e4172c43ed28b2339ce7a209"
server_key_iv: "1ebdf286bf480240e0d67d8b9261b13d"
server_key_version: "encryptionkey@0"
I20260808 15:07:19.163086 30756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "e563a0c5c36a44efad60dcefafd511d2"
format_stamp: "Formatted at 2026-08-08 15:07:19 on dist-test-slave-jlzn"
server_key: "69b12100e4172c43ed28b2339ce7a209"
server_key_iv: "1ebdf286bf480240e0d67d8b9261b13d"
server_key_version: "encryptionkey@0"
I20260808 15:07:19.167972 30756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 15:07:19.171182 30775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:19.172672 30756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:07:19.172811 30756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "e563a0c5c36a44efad60dcefafd511d2"
format_stamp: "Formatted at 2026-08-08 15:07:19 on dist-test-slave-jlzn"
server_key: "69b12100e4172c43ed28b2339ce7a209"
server_key_iv: "1ebdf286bf480240e0d67d8b9261b13d"
server_key_version: "encryptionkey@0"
I20260808 15:07:19.173000 30756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:19.194888 30756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:19.195591 30756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:19.195883 30756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:19.196749 30756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:19.198062 30756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:19.198165 30756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:19.198268 30756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:19.198338 30756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:19.226655 30756 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:45881
I20260808 15:07:19.226728 30946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:45881 every 8 connection(s)
I20260808 15:07:19.227825 30756 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:07:19.228637 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 30756
I20260808 15:07:19.228793 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:07:19.229290 2771 external_mini_cluster.cc:1428] Setting key 439b0b2ace3d0669c7029819b6cd8823
I20260808 15:07:19.232344 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46301
--builtin_ntp_servers=127.2.180.212:33671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:07:19.239642 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:19.230368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:40579 (local address 127.2.180.254:46301)
0808 15:07:19.230854 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:19.230860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:19.231961 (+ 1101us) server_negotiation.cc:407] Connection header received
0808 15:07:19.233708 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:19.233714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:19.233793 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:19.233885 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:19.234868 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:19.235580 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:19.237098 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:19.237503 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:19.237611 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:19.238819 (+ 1208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:19.238824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:19.238827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:19.238843 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:19.238885 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:19.238897 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:19.238969 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:19.239201 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 15:07:19.239299 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":225,"threads_started":1}
I20260808 15:07:19.240504 30951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:19.230934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46301 (local address 127.2.180.193:40579)
0808 15:07:19.231792 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:19.231866 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:19.233463 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:19.234110 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:19.234118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:19.234198 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:19.234713 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:19.234729 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:19.235742 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:19.235746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:19.236965 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:19.236977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:19.237720 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:19.237728 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:19.237943 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:19.238530 (+ 587us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:19.238533 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:19.238567 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:19.238640 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:19.238984 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:19.238995 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:19.239003 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:07:19.239144 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 15:07:19.239488 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":294,"threads_started":1}
I20260808 15:07:19.242307 30947 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46301
I20260808 15:07:19.242594 30947 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:19.243285 30947 heartbeater.cc:511] Master 127.2.180.254:46301 requested a full tablet report, sending...
I20260808 15:07:19.245420 30679 ts_manager.cc:194] Registered new tserver with Master: e563a0c5c36a44efad60dcefafd511d2 (127.2.180.193:45881)
I20260808 15:07:19.247326 30679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.193:40579
W20260808 15:07:19.550796 30952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:19.551044 30952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:19.551129 30952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:19.557997 30952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:19.558084 30952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:19.558187 30952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:07:19.566608 30952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:19.567667 30952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:19.569274 30952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:19.581497 30966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:19.581598 30960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:19.581571 30961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:19.582353 30952 server_base.cc:1034] running on GCE node
I20260808 15:07:19.582856 30952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:19.583657 30952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:19.584848 30952 hybrid_clock.cc:648] HybridClock initialized: now 1786201639584796 us; error 43 us; skew 500 ppm
I20260808 15:07:19.585134 30952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:19.587114 30952 webserver.cc:460] Webserver started at http://127.2.180.194:45903/ using document root <none> and password file <none>
I20260808 15:07:19.587725 30952 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:19.587832 30952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:19.588128 30952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:19.590751 30952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "6e49c2fc42d3424dafc4f970e431d035"
format_stamp: "Formatted at 2026-08-08 15:07:19 on dist-test-slave-jlzn"
server_key: "b4b8d7dd0e9c19a090ac5d8185b3974a"
server_key_iv: "911558866026929f23011561ceba350a"
server_key_version: "encryptionkey@0"
I20260808 15:07:19.591480 30952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "6e49c2fc42d3424dafc4f970e431d035"
format_stamp: "Formatted at 2026-08-08 15:07:19 on dist-test-slave-jlzn"
server_key: "b4b8d7dd0e9c19a090ac5d8185b3974a"
server_key_iv: "911558866026929f23011561ceba350a"
server_key_version: "encryptionkey@0"
I20260808 15:07:19.596122 30952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260808 15:07:19.599208 30974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:19.600422 30952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:07:19.600566 30952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "6e49c2fc42d3424dafc4f970e431d035"
format_stamp: "Formatted at 2026-08-08 15:07:19 on dist-test-slave-jlzn"
server_key: "b4b8d7dd0e9c19a090ac5d8185b3974a"
server_key_iv: "911558866026929f23011561ceba350a"
server_key_version: "encryptionkey@0"
I20260808 15:07:19.600754 30952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:19.613337 30952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:19.614012 30952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:19.614280 30952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:19.615125 30952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:19.616448 30952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:19.616542 30952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:19.616649 30952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:19.616721 30952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:19.645226 30952 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:35599
I20260808 15:07:19.645264 31149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:35599 every 8 connection(s)
I20260808 15:07:19.646412 30952 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:07:19.650076 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 30952
I20260808 15:07:19.650245 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:07:19.650708 2771 external_mini_cluster.cc:1428] Setting key 9e92fdf724b6338aba8677abaf99bd60
I20260808 15:07:19.653466 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46301
--builtin_ntp_servers=127.2.180.212:33671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:07:19.658097 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:19.648667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:45893 (local address 127.2.180.254:46301)
0808 15:07:19.648833 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:19.648838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:19.650108 (+ 1270us) server_negotiation.cc:407] Connection header received
0808 15:07:19.651945 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:19.651950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:19.652013 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:19.652091 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:19.653117 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:19.654085 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:19.655555 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:19.656134 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:19.656260 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:19.657402 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:19.657408 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:19.657412 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:19.657431 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:19.657466 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:19.657481 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:19.657564 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:19.657726 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 15:07:19.657812 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:07:19.659121 31155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:19.649114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46301 (local address 127.2.180.194:45893)
0808 15:07:19.649958 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:19.649991 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:19.651658 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:19.652344 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:19.652351 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:19.652429 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:19.652956 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:19.652972 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:19.654233 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:19.654237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:19.655415 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:19.655428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:19.656318 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:19.656323 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:19.656509 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:19.657134 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:19.657137 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:19.657172 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:19.657238 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:19.657565 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:19.657571 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:19.657576 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 15:07:19.657699 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 15:07:19.657948 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":262,"threads_started":1}
I20260808 15:07:19.660367 31150 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46301
I20260808 15:07:19.660621 31150 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:19.661298 31150 heartbeater.cc:511] Master 127.2.180.254:46301 requested a full tablet report, sending...
I20260808 15:07:19.662550 30679 ts_manager.cc:194] Registered new tserver with Master: 6e49c2fc42d3424dafc4f970e431d035 (127.2.180.194:35599)
I20260808 15:07:19.663309 30679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.194:45893
W20260808 15:07:19.975821 31157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:19.976060 31157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:19.976146 31157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:19.983057 31157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:19.983140 31157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:19.983242 31157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:07:19.991628 31157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:19.992686 31157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:19.994240 31157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:20.006261 31164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:20.006220 31169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:20.006297 31166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:20.007138 31157 server_base.cc:1034] running on GCE node
I20260808 15:07:20.007731 31157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:20.008589 31157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:20.009788 31157 hybrid_clock.cc:648] HybridClock initialized: now 1786201640009745 us; error 30 us; skew 500 ppm
I20260808 15:07:20.010076 31157 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:20.012264 31157 webserver.cc:460] Webserver started at http://127.2.180.195:46193/ using document root <none> and password file <none>
I20260808 15:07:20.012892 31157 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:20.012991 31157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:20.013288 31157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:20.015920 31157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "3fe150f610d241e8ad6cc205f16dcc5d"
format_stamp: "Formatted at 2026-08-08 15:07:20 on dist-test-slave-jlzn"
server_key: "2edeff276779738eb18489c347470463"
server_key_iv: "8791d0ad46a0480b6ce88469d1a825d6"
server_key_version: "encryptionkey@0"
I20260808 15:07:20.016676 31157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "3fe150f610d241e8ad6cc205f16dcc5d"
format_stamp: "Formatted at 2026-08-08 15:07:20 on dist-test-slave-jlzn"
server_key: "2edeff276779738eb18489c347470463"
server_key_iv: "8791d0ad46a0480b6ce88469d1a825d6"
server_key_version: "encryptionkey@0"
I20260808 15:07:20.021200 31157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 15:07:20.024456 31177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:20.025617 31157 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:07:20.025771 31157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "3fe150f610d241e8ad6cc205f16dcc5d"
format_stamp: "Formatted at 2026-08-08 15:07:20 on dist-test-slave-jlzn"
server_key: "2edeff276779738eb18489c347470463"
server_key_iv: "8791d0ad46a0480b6ce88469d1a825d6"
server_key_version: "encryptionkey@0"
I20260808 15:07:20.025964 31157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:20.039700 31157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:20.040441 31157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:20.040704 31157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:20.041550 31157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:20.042836 31157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:20.042932 31157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:20.043042 31157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:20.043116 31157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:20.072286 31157 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:40739
I20260808 15:07:20.072340 31341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:40739 every 8 connection(s)
I20260808 15:07:20.073459 31157 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:07:20.079416 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 31157
I20260808 15:07:20.079594 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:07:20.080152 2771 external_mini_cluster.cc:1428] Setting key 04f4d50d4d5359a49baea3e96d6d2e49
I20260808 15:07:20.085127 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.075633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:42557 (local address 127.2.180.254:46301)
0808 15:07:20.075791 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.075795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.077110 (+ 1315us) server_negotiation.cc:407] Connection header received
0808 15:07:20.078792 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.078796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.078865 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.078953 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:20.079974 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.080930 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.082919 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.083320 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.083413 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.084523 (+ 1110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:20.084528 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:20.084531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:20.084547 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:20.084577 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:20.084594 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:20.084662 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.084820 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.084891 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 15:07:20.085939 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.076099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46301 (local address 127.2.180.195:42557)
0808 15:07:20.076960 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.076999 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:20.078565 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:20.079120 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:20.079127 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:20.079205 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:20.079764 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.079782 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.081108 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.081113 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.082785 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.082796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.083489 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.083494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.083681 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.084258 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:20.084261 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:20.084293 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:20.084358 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:20.084667 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:20.084673 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:20.084677 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 15:07:20.084794 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 15:07:20.085034 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":256,"threads_started":1}
I20260808 15:07:20.087085 31344 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46301
I20260808 15:07:20.087330 31344 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:20.088071 31344 heartbeater.cc:511] Master 127.2.180.254:46301 requested a full tablet report, sending...
I20260808 15:07:20.089272 30679 ts_manager.cc:194] Registered new tserver with Master: 3fe150f610d241e8ad6cc205f16dcc5d (127.2.180.195:40739)
I20260808 15:07:20.089986 30679 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.195:42557
I20260808 15:07:20.094751 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:07:20.103617 30950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.098486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55014 (local address 127.2.180.254:46301)
0808 15:07:20.098717 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.098722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.099050 (+ 328us) server_negotiation.cc:407] Connection header received
0808 15:07:20.099178 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.099182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.099245 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.099312 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:20.099732 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.100387 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.101668 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.102022 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.102104 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.102995 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:20.102999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:20.103003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:20.103019 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:20.103047 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:20.103058 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:20.103169 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.103363 (+ 194us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.103412 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:07:20.106853 30679 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:07:20.113391 30679 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:07:20.134399 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.123630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:40739 (local address 127.0.0.1:35760)
0808 15:07:20.124421 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.124456 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:20.124612 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:20.126210 (+ 1598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:20.126218 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:20.126250 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:20.126653 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.126665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.129010 (+ 2345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.129016 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.131486 (+ 2470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.131514 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.133491 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.133499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.133723 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.133775 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 15:07:20.133873 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 15:07:20.134051 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":371,"threads_started":1}
I20260808 15:07:20.139601 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.123914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35760 (local address 127.2.180.195:40739)
0808 15:07:20.125598 (+ 1684us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.125606 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.125642 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:07:20.125734 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.125742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.125958 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.126887 (+ 929us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:20.127146 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.128845 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.131793 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.133253 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.138771 (+ 5518us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.139122 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.139202 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.139329 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1535,"thread_start_us":1397,"threads_started":1}
I20260808 15:07:20.142186 31362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.124029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:45881 (local address 127.0.0.1:41330)
0808 15:07:20.134643 (+ 10614us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.134660 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:20.134763 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:20.135684 (+ 921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:20.135690 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:20.135716 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:20.135895 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.135905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.137855 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.137860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.139687 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.139702 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.141451 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.141457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.141675 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.141707 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:07:20.141796 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 15:07:20.141924 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10499,"thread_start_us":12973,"threads_started":1}
I20260808 15:07:20.142360 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.124580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.2.180.193:45881)
0808 15:07:20.125168 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.125177 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.134889 (+ 9712us) server_negotiation.cc:407] Connection header received
0808 15:07:20.134998 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.135006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.135249 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.135425 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:20.136071 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.137690 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.139936 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.141266 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.141396 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.141613 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.141858 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.142021 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":282,"threads_started":1}
I20260808 15:07:20.143277 31248 tablet_service.cc:1467] Processing CreateTablet for tablet 81fd49c42b75424eade731c10d50adac (DEFAULT_TABLE table=test-table [id=dc167a79a74741a4868ae3f49eb5aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:20.144754 31248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fd49c42b75424eade731c10d50adac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:20.145687 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.124937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:35599 (local address 127.0.0.1:41046)
0808 15:07:20.136798 (+ 11861us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.136816 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:20.136929 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:20.138306 (+ 1377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:20.138315 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:20.138373 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:20.138819 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.138833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.140726 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.140732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.142879 (+ 2147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.142891 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.144899 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.144906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.145125 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.145166 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 15:07:20.145249 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 15:07:20.145362 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11733,"mutex_wait_us":1,"thread_start_us":14099,"threads_started":1}
I20260808 15:07:20.146266 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.125220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41046 (local address 127.2.180.194:35599)
0808 15:07:20.137572 (+ 12352us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.137585 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.137607 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:07:20.137680 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.137685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.137859 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.138042 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:20.139299 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.140548 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.143041 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.144392 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.144520 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.144760 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.145483 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.145657 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12207,"thread_start_us":12088,"threads_started":1}
I20260808 15:07:20.145869 30848 tablet_service.cc:1467] Processing CreateTablet for tablet 81fd49c42b75424eade731c10d50adac (DEFAULT_TABLE table=test-table [id=dc167a79a74741a4868ae3f49eb5aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:20.146940 30848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fd49c42b75424eade731c10d50adac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:20.149739 31050 tablet_service.cc:1467] Processing CreateTablet for tablet 81fd49c42b75424eade731c10d50adac (DEFAULT_TABLE table=test-table [id=dc167a79a74741a4868ae3f49eb5aba3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:20.151005 31050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81fd49c42b75424eade731c10d50adac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:20.155905 31373 tablet_bootstrap.cc:492] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Bootstrap starting.
I20260808 15:07:20.159134 31373 tablet_bootstrap.cc:654] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:20.160200 31373 log.cc:824] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:20.161578 31374 tablet_bootstrap.cc:492] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: Bootstrap starting.
I20260808 15:07:20.163314 31373 tablet_bootstrap.cc:492] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: No bootstrap required, opened a new log
I20260808 15:07:20.163561 31373 ts_tablet_manager.cc:1397] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 15:07:20.164011 31376 tablet_bootstrap.cc:492] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Bootstrap starting.
I20260808 15:07:20.164996 31374 tablet_bootstrap.cc:654] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:20.166150 31374 log.cc:824] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:20.166657 31376 tablet_bootstrap.cc:654] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:20.167755 31376 log.cc:824] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:20.169220 31374 tablet_bootstrap.cc:492] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: No bootstrap required, opened a new log
I20260808 15:07:20.169478 31374 ts_tablet_manager.cc:1397] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260808 15:07:20.170112 31376 tablet_bootstrap.cc:492] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: No bootstrap required, opened a new log
I20260808 15:07:20.170322 31376 ts_tablet_manager.cc:1397] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 15:07:20.174207 31373 raft_consensus.cc:348] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.174618 31373 raft_consensus.cc:374] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:20.174723 31373 raft_consensus.cc:729] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fe150f610d241e8ad6cc205f16dcc5d, State: Initialized, Role: FOLLOWER
I20260808 15:07:20.175275 31373 consensus_queue.cc:260] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.176378 31344 heartbeater.cc:503] Master 127.2.180.254:46301 was elected leader, sending a full tablet report...
I20260808 15:07:20.178133 31373 ts_tablet_manager.cc:1428] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Time spent starting tablet: real 0.014s user 0.007s sys 0.007s
I20260808 15:07:20.178467 31376 raft_consensus.cc:348] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.178857 31376 raft_consensus.cc:374] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:20.178954 31376 raft_consensus.cc:729] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e49c2fc42d3424dafc4f970e431d035, State: Initialized, Role: FOLLOWER
I20260808 15:07:20.179031 31374 raft_consensus.cc:348] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.179374 31374 raft_consensus.cc:374] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:20.179467 31374 raft_consensus.cc:729] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e563a0c5c36a44efad60dcefafd511d2, State: Initialized, Role: FOLLOWER
I20260808 15:07:20.179482 31376 consensus_queue.cc:260] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.179984 31374 consensus_queue.cc:260] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.180749 31150 heartbeater.cc:503] Master 127.2.180.254:46301 was elected leader, sending a full tablet report...
I20260808 15:07:20.181319 31376 ts_tablet_manager.cc:1428] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260808 15:07:20.184442 30947 heartbeater.cc:503] Master 127.2.180.254:46301 was elected leader, sending a full tablet report...
I20260808 15:07:20.184952 31374 ts_tablet_manager.cc:1428] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
W20260808 15:07:20.229796 30949 tablet.cc:2369] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:07:20.325001 31346 tablet.cc:2369] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:07:20.338446 31383 raft_consensus.cc:482] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:07:20.338671 31383 raft_consensus.cc:504] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.340349 31383 leader_election.cc:288] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fe150f610d241e8ad6cc205f16dcc5d (127.2.180.195:40739), 6e49c2fc42d3424dafc4f970e431d035 (127.2.180.194:35599)
I20260808 15:07:20.347882 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.340875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:47035 (local address 127.2.180.194:35599)
0808 15:07:20.341062 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.341068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.341724 (+ 656us) server_negotiation.cc:407] Connection header received
0808 15:07:20.341881 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.341887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.341964 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.342060 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:20.342724 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.343894 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.345960 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.347052 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.347221 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.347291 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.347499 (+ 208us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.347628 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:07:20.347975 31385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.340715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:35599 (local address 127.2.180.193:47035)
0808 15:07:20.341626 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.341639 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:20.341768 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:20.342081 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:20.342086 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:20.342132 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:20.342546 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.342563 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.344020 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.344028 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.345778 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.345792 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.347236 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.347241 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.347349 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.347373 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 15:07:20.347443 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 15:07:20.347552 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"mutex_wait_us":91,"spinlock_wait_cycles":48896,"thread_start_us":503,"threads_started":1}
I20260808 15:07:20.348280 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.340873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:57451 (local address 127.2.180.195:40739)
0808 15:07:20.341062 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.341068 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.341694 (+ 626us) server_negotiation.cc:407] Connection header received
0808 15:07:20.341881 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.341887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.341965 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.342060 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:20.342783 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.343854 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.346200 (+ 2346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.347407 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.347572 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.347634 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.347922 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.348052 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:07:20.348461 31388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.340590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:40739 (local address 127.2.180.193:57451)
0808 15:07:20.341570 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.341589 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:20.341752 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:20.342081 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:20.342085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:20.342129 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:20.342658 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.342669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.343974 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.343979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.345998 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:20.346014 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.347586 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.347592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:20.347756 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.347790 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 15:07:20.347895 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 15:07:20.348016 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":122,"spinlock_wait_cycles":14976,"thread_start_us":640,"threads_started":1}
I20260808 15:07:20.349109 31081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fd49c42b75424eade731c10d50adac" candidate_uuid: "e563a0c5c36a44efad60dcefafd511d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e49c2fc42d3424dafc4f970e431d035" is_pre_election: true
I20260808 15:07:20.349412 31273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fd49c42b75424eade731c10d50adac" candidate_uuid: "e563a0c5c36a44efad60dcefafd511d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" is_pre_election: true
I20260808 15:07:20.349573 31081 raft_consensus.cc:2454] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e563a0c5c36a44efad60dcefafd511d2 in term 0.
I20260808 15:07:20.349858 31273 raft_consensus.cc:2454] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e563a0c5c36a44efad60dcefafd511d2 in term 0.
I20260808 15:07:20.350150 30778 leader_election.cc:302] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e49c2fc42d3424dafc4f970e431d035, e563a0c5c36a44efad60dcefafd511d2; no voters:
I20260808 15:07:20.350572 31383 raft_consensus.cc:2781] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:07:20.350700 31383 raft_consensus.cc:482] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:07:20.350800 31383 raft_consensus.cc:3037] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:20.353024 31383 raft_consensus.cc:504] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.353601 31383 leader_election.cc:288] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [CANDIDATE]: Term 1 election: Requested vote from peers 3fe150f610d241e8ad6cc205f16dcc5d (127.2.180.195:40739), 6e49c2fc42d3424dafc4f970e431d035 (127.2.180.194:35599)
I20260808 15:07:20.353935 31273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fd49c42b75424eade731c10d50adac" candidate_uuid: "e563a0c5c36a44efad60dcefafd511d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fe150f610d241e8ad6cc205f16dcc5d"
I20260808 15:07:20.354017 31081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81fd49c42b75424eade731c10d50adac" candidate_uuid: "e563a0c5c36a44efad60dcefafd511d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e49c2fc42d3424dafc4f970e431d035"
I20260808 15:07:20.354099 31273 raft_consensus.cc:3037] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:20.354198 31081 raft_consensus.cc:3037] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:20.356226 31273 raft_consensus.cc:2454] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e563a0c5c36a44efad60dcefafd511d2 in term 1.
I20260808 15:07:20.356407 31081 raft_consensus.cc:2454] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e563a0c5c36a44efad60dcefafd511d2 in term 1.
I20260808 15:07:20.356570 30779 leader_election.cc:302] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fe150f610d241e8ad6cc205f16dcc5d, e563a0c5c36a44efad60dcefafd511d2; no voters:
I20260808 15:07:20.356854 31383 raft_consensus.cc:2781] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:20.357275 31383 raft_consensus.cc:686] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 1 LEADER]: Becoming Leader. State: Replica: e563a0c5c36a44efad60dcefafd511d2, State: Running, Role: LEADER
I20260808 15:07:20.357698 31383 consensus_queue.cc:237] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } }
I20260808 15:07:20.361989 30677 catalog_manager.cc:5495] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to e563a0c5c36a44efad60dcefafd511d2 (127.2.180.193). New cstate: current_term: 1 leader_uuid: "e563a0c5c36a44efad60dcefafd511d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e563a0c5c36a44efad60dcefafd511d2" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 45881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 } health_report { overall_health: UNKNOWN } } }
W20260808 15:07:20.398150 31152 tablet.cc:2369] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:07:20.428359 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.423104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41342 (local address 127.2.180.193:45881)
0808 15:07:20.423307 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.423314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.423400 (+ 86us) server_negotiation.cc:407] Connection header received
0808 15:07:20.423605 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.423609 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.423701 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.423862 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:07:20.424224 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.424903 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.426236 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.426636 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.426742 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.427311 (+ 569us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:07:20.427769 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:07:20.427949 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.428084 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.428126 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:07:20.435766 31081 raft_consensus.cc:1261] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 1 FOLLOWER]: Refusing update from remote peer e563a0c5c36a44efad60dcefafd511d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:20.435765 31273 raft_consensus.cc:1261] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 1 FOLLOWER]: Refusing update from remote peer e563a0c5c36a44efad60dcefafd511d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:20.436612 31389 consensus_queue.cc:1035] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e49c2fc42d3424dafc4f970e431d035" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 35599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:20.436925 31383 consensus_queue.cc:1035] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fe150f610d241e8ad6cc205f16dcc5d" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 40739 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:20.458877 30679 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:20.459116 30679 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:20.462396 30679 catalog_manager.cc:5782] T 00000000000000000000000000000000 P edc7f6728517427db8c44d7980fd6178: Sending DeleteTablet for 3 replicas of tablet 81fd49c42b75424eade731c10d50adac
I20260808 15:07:20.463423 31248 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fd49c42b75424eade731c10d50adac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:20 UTC) from {username='slave'} at 127.0.0.1:35760
I20260808 15:07:20.463510 30848 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fd49c42b75424eade731c10d50adac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:20 UTC) from {username='slave'} at 127.0.0.1:41330
I20260808 15:07:20.463722 31050 tablet_service.cc:1514] Processing DeleteTablet for tablet 81fd49c42b75424eade731c10d50adac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:20 UTC) from {username='slave'} at 127.0.0.1:41046
I20260808 15:07:20.465554 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 30756
I20260808 15:07:20.475490 31415 tablet_replica.cc:330] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: stopping tablet replica
I20260808 15:07:20.475908 31415 raft_consensus.cc:2229] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:20.476356 31415 raft_consensus.cc:2258] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:20.478493 31415 ts_tablet_manager.cc:1905] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:20.481292 31410 tablet_replica.cc:330] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: stopping tablet replica
I20260808 15:07:20.481657 31410 raft_consensus.cc:2229] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:07:20.482028 31413 tablet_replica.cc:330] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: stopping tablet replica
I20260808 15:07:20.482275 31410 raft_consensus.cc:2258] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:20.482404 31413 raft_consensus.cc:2229] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:20.483053 31413 raft_consensus.cc:2258] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:20.485188 31413 ts_tablet_manager.cc:1905] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:20.488624 31410 ts_tablet_manager.cc:1905] T 81fd49c42b75424eade731c10d50adac P e563a0c5c36a44efad60dcefafd511d2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:20.489328 31366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.466806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41356 (local address 127.2.180.193:45881)
0808 15:07:20.466974 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.466980 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.481467 (+ 14487us) server_negotiation.cc:407] Connection header received
0808 15:07:20.481668 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.481674 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.481773 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.481876 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:20.482660 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.483732 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.485568 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.486173 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.486286 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.487644 (+ 1358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:20.487669 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:20.487683 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:20.488407 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:20.488464 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:20.488488 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:20.488585 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.488841 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.488939 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:07:20.490546 30941 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:20.491184 31415 ts_tablet_manager.cc:1918] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:20.491344 31415 log.cc:1196] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/81fd49c42b75424eade731c10d50adac
I20260808 15:07:20.491374 31413 ts_tablet_manager.cc:1918] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:20.491539 31413 log.cc:1196] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/81fd49c42b75424eade731c10d50adac
I20260808 15:07:20.491946 31415 ts_tablet_manager.cc:1939] T 81fd49c42b75424eade731c10d50adac P 6e49c2fc42d3424dafc4f970e431d035: Deleting consensus metadata
I20260808 15:07:20.491998 31413 ts_tablet_manager.cc:1939] T 81fd49c42b75424eade731c10d50adac P 3fe150f610d241e8ad6cc205f16dcc5d: Deleting consensus metadata
I20260808 15:07:20.493350 30661 catalog_manager.cc:4841] TS 3fe150f610d241e8ad6cc205f16dcc5d (127.2.180.195:40739): tablet 81fd49c42b75424eade731c10d50adac (table test-table [id=dc167a79a74741a4868ae3f49eb5aba3]) successfully deleted
I20260808 15:07:20.493662 30658 catalog_manager.cc:4841] TS 6e49c2fc42d3424dafc4f970e431d035 (127.2.180.194:35599): tablet 81fd49c42b75424eade731c10d50adac (table test-table [id=dc167a79a74741a4868ae3f49eb5aba3]) successfully deleted
I20260808 15:07:20.588191 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 30756
I20260808 15:07:20.612418 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 30952
W20260808 15:07:20.612439 30656 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.180.193:45881 (error 108)
I20260808 15:07:20.614053 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.613640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:45881 (local address 127.0.0.1:41362)
0808 15:07:20.613800 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.613909 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 15:07:20.614285 30656 catalog_manager.cc:4584] TS e563a0c5c36a44efad60dcefafd511d2 (127.2.180.193:45881): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 81fd49c42b75424eade731c10d50adac: Network error: Client connection negotiation failed: client connection to 127.2.180.193:45881: connect: Connection refused (error 111)
I20260808 15:07:20.618956 31372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.613201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.2.180.194:35599)
0808 15:07:20.613378 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.613382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.613493 (+ 111us) server_negotiation.cc:407] Connection header received
0808 15:07:20.613692 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.613698 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.613825 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.613922 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:20.614418 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.615217 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.616544 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.616940 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.617039 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.617868 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:20.617877 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:20.617890 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:20.618437 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:20.618483 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:20.618503 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:20.618590 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.618652 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.618733 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:07:20.619603 31143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:20.667637 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.667259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:45881 (local address 127.0.0.1:41372)
0808 15:07:20.667416 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.667494 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 15:07:20.668558 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.668242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:45881 (local address 127.0.0.1:41386)
0808 15:07:20.668368 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.668436 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 15:07:20.704483 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 30952
I20260808 15:07:20.729018 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.728522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:45881 (local address 127.0.0.1:41390)
0808 15:07:20.728734 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.728839 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 15:07:20.730053 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.729683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:45881 (local address 127.0.0.1:41406)
0808 15:07:20.729837 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.729922 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 15:07:20.731360 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 31157
I20260808 15:07:20.737177 31363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.732002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.2.180.195:40739)
0808 15:07:20.732148 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:20.732153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:20.732172 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:07:20.732313 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:20.732316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:20.732399 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:20.732479 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:20.732809 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.733500 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.734695 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:20.735065 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:20.735150 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:20.736068 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:20.736077 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:20.736094 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:20.736636 (+ 542us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:20.736694 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:20.736714 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:20.736793 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:20.736865 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 15:07:20.736948 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 15:07:20.737771 31336 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:20.810591 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.810103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:45881 (local address 127.0.0.1:41418)
0808 15:07:20.810312 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.810422 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 15:07:20.811640 31365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:20.811242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:45881 (local address 127.0.0.1:41426)
0808 15:07:20.811447 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:20.811526 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:45881: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 15:07:20.824968 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 31157
I20260808 15:07:20.844260 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 30635
I20260808 15:07:20.845304 30718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:20.953109 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 30635
2026-08-08T15:07:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2728 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:21 dist-test-slave-jlzn krb5kdc[31428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:21 dist-test-slave-jlzn krb5kdc[31428](info): set up 2 sockets
Aug 08 15:07:21 dist-test-slave-jlzn krb5kdc[31428](info): commencing operation
krb5kdc: starting...
W20260808 15:07:22.135527 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.009s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:22 dist-test-slave-jlzn krb5kdc[31428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:07:22.376351 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46269
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:46269
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:22.703867 31441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:22.704193 31441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:22.704293 31441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:22.711534 31441 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:22.711630 31441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:22.711694 31441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:22.711750 31441 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:22.711791 31441 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:22.721519 31441 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:46269
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:46269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:22.722652 31441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:22.723907 31441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:22.735234 31453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:22.735070 31450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:22.735584 31448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:22.736577 31441 server_base.cc:1034] running on GCE node
I20260808 15:07:22.737325 31441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:22.738569 31441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:22.739770 31441 hybrid_clock.cc:648] HybridClock initialized: now 1786201642739724 us; error 38 us; skew 500 ppm
I20260808 15:07:22.740108 31441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:22 dist-test-slave-jlzn krb5kdc[31428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201642, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:22.742926 31441 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:07:22.744266 31441 webserver.cc:460] Webserver started at http://127.2.180.254:40861/ using document root <none> and password file <none>
I20260808 15:07:22.744881 31441 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:22.744999 31441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:22.745306 31441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:22.748178 31441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee"
format_stamp: "Formatted at 2026-08-08 15:07:22 on dist-test-slave-jlzn"
server_key: "4bc86d1bf38f2568021eaf59af1857bf"
server_key_iv: "f4dce089bc34582141be2235e1cdfbf8"
server_key_version: "encryptionkey@0"
I20260808 15:07:22.748903 31441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee"
format_stamp: "Formatted at 2026-08-08 15:07:22 on dist-test-slave-jlzn"
server_key: "4bc86d1bf38f2568021eaf59af1857bf"
server_key_iv: "f4dce089bc34582141be2235e1cdfbf8"
server_key_version: "encryptionkey@0"
I20260808 15:07:22.753703 31441 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:07:22.756880 31461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:22.758098 31441 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.003s
I20260808 15:07:22.758256 31441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee"
format_stamp: "Formatted at 2026-08-08 15:07:22 on dist-test-slave-jlzn"
server_key: "4bc86d1bf38f2568021eaf59af1857bf"
server_key_iv: "f4dce089bc34582141be2235e1cdfbf8"
server_key_version: "encryptionkey@0"
I20260808 15:07:22.758430 31441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:22.773157 31441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:22.776810 31441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:22.777052 31441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:22.793725 31441 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:46269
I20260808 15:07:22.793743 31539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:46269 every 8 connection(s)
I20260808 15:07:22.794906 31441 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:22.796252 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 31441
I20260808 15:07:22.796414 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:22.796833 2771 external_mini_cluster.cc:1428] Setting key 61e24731d9a50f422834857385327d95
I20260808 15:07:22.799355 31540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 15:07:22 dist-test-slave-jlzn krb5kdc[31428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:22.811741 31540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee: Bootstrap starting.
I20260808 15:07:22.814973 31540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:22.816111 31540 log.cc:824] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:22.819339 31540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee: No bootstrap required, opened a new log
I20260808 15:07:22.819941 31543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:22.798438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42186 (local address 127.2.180.254:46269)
0808 15:07:22.799171 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:22.799181 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:22.799218 (+ 37us) server_negotiation.cc:407] Connection header received
0808 15:07:22.800878 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:22.800898 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:22.801225 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:22.801520 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:22.803375 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:22.805193 (+ 1818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:22.807014 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:22.808018 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:22.808187 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:22.812497 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:22.812504 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:22.812517 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:22.812559 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:22.815510 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:22.816361 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:22.816368 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:22.816375 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:22.816586 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:22.817058 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:22.817061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:22.817063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:22.817617 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:22.817883 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:22.818238 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 15:07:22.818626 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":319,"threads_started":1}
I20260808 15:07:22.826490 31540 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46269 } }
I20260808 15:07:22.826793 31540 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:22.826874 31540 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4cf7fbf1c9c4ee890ce74f391f7cfee, State: Initialized, Role: FOLLOWER
I20260808 15:07:22.827309 31540 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46269 } }
I20260808 15:07:22.827508 31540 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:22.827602 31540 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:22.827725 31540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:22.829687 31540 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46269 } }
I20260808 15:07:22.830088 31540 leader_election.cc:302] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4cf7fbf1c9c4ee890ce74f391f7cfee; no voters:
I20260808 15:07:22.830601 31540 leader_election.cc:288] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:22.830715 31547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:22.831768 31547 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [term 1 LEADER]: Becoming Leader. State: Replica: a4cf7fbf1c9c4ee890ce74f391f7cfee, State: Running, Role: LEADER
I20260808 15:07:22.832275 31547 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46269 } }
I20260808 15:07:22.832531 31540 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:22.834771 31551 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4cf7fbf1c9c4ee890ce74f391f7cfee. Latest consensus state: current_term: 1 leader_uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46269 } } }
I20260808 15:07:22.835156 31551 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:22.835451 31549 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4cf7fbf1c9c4ee890ce74f391f7cfee" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 46269 } } }
I20260808 15:07:22.835740 31549 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:22.837555 31560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:22.842427 31560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:22.850809 31560 catalog_manager.cc:1347] Generated new cluster ID: cbd218445afc40d88348d993f426a046
I20260808 15:07:22.850898 31560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:22.877594 31560 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:22.879213 31560 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:22.886441 31560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee: Generated new TSK 0
I20260808 15:07:22.887341 31560 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.2.180.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
Entry for principal kudu/127.2.180.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab.
I20260808 15:07:22.944435 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46269
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:23.263247 31575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:23.263490 31575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:23.263584 31575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:23.270574 31575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:23.270674 31575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:23.270784 31575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:07:23.279961 31575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.193.keytab
--principal=kudu/127.2.180.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:23.281068 31575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:23.282212 31575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:23.294344 31587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:23.294379 31592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:23.294423 31589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:23.295488 31575 server_base.cc:1034] running on GCE node
I20260808 15:07:23.296103 31575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:23.296906 31575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:23.298105 31575 hybrid_clock.cc:648] HybridClock initialized: now 1786201643298063 us; error 42 us; skew 500 ppm
I20260808 15:07:23.298388 31575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:23 dist-test-slave-jlzn krb5kdc[31428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201643, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:23.301365 31575 init.cc:377] Logged in from keytab as kudu/127.2.180.193@KRBTEST.COM (short username kudu)
I20260808 15:07:23.302711 31575 webserver.cc:460] Webserver started at http://127.2.180.193:33131/ using document root <none> and password file <none>
I20260808 15:07:23.303318 31575 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:23.303431 31575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:23.303726 31575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:23.306473 31575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "0d165294cac04ca8b304deaa25641447"
format_stamp: "Formatted at 2026-08-08 15:07:23 on dist-test-slave-jlzn"
server_key: "3bc99ba41d8047f2b60fbe3ecd7321a4"
server_key_iv: "e6cd0acfbbe0a825137f2cb89a140a37"
server_key_version: "encryptionkey@0"
I20260808 15:07:23.307224 31575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "0d165294cac04ca8b304deaa25641447"
format_stamp: "Formatted at 2026-08-08 15:07:23 on dist-test-slave-jlzn"
server_key: "3bc99ba41d8047f2b60fbe3ecd7321a4"
server_key_iv: "e6cd0acfbbe0a825137f2cb89a140a37"
server_key_version: "encryptionkey@0"
I20260808 15:07:23.311976 31575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 15:07:23.315060 31605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:23.316339 31575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:07:23.316500 31575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "0d165294cac04ca8b304deaa25641447"
format_stamp: "Formatted at 2026-08-08 15:07:23 on dist-test-slave-jlzn"
server_key: "3bc99ba41d8047f2b60fbe3ecd7321a4"
server_key_iv: "e6cd0acfbbe0a825137f2cb89a140a37"
server_key_version: "encryptionkey@0"
I20260808 15:07:23.316681 31575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:23.335243 31575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:23.338908 31575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:23.339160 31575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:23.340049 31575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:23.341279 31575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:23.341356 31575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:23.341475 31575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:23.341528 31575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:23.370424 31575 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:36087
I20260808 15:07:23.370479 31767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:36087 every 8 connection(s)
I20260808 15:07:23.371591 31575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:07:23.375741 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 31575
I20260808 15:07:23.375938 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:07:23.376415 2771 external_mini_cluster.cc:1428] Setting key 11e3b18e37aa6dd89c259414e7590b8e
Aug 08 15:07:23 dist-test-slave-jlzn krb5kdc[31428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201643, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.193@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:23.390669 31770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:23.373897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:53863 (local address 127.2.180.254:46269)
0808 15:07:23.374338 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:23.374344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:23.375241 (+ 897us) server_negotiation.cc:407] Connection header received
0808 15:07:23.377071 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:23.377077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:23.377161 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:23.377260 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:23.378637 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:23.379344 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:23.380856 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:23.381262 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:23.381372 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:23.384870 (+ 3498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:23.384875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:23.384878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:23.384909 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:23.387577 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:23.388392 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:23.388395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:23.388398 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:23.388513 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:23.388946 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:23.388950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:23.388952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:23.389224 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:23.389332 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:23.390086 (+ 754us) server_negotiation.cc:299] Negotiation successful
0808 15:07:23.390309 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":209,"threads_started":1}
I20260808 15:07:23.391295 31771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:23.374424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46269 (local address 127.2.180.193:53863)
0808 15:07:23.375085 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:23.375124 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:23.376819 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:23.377490 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:23.377498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:23.377975 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:23.378478 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:23.378494 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:23.379494 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:23.379498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:23.380721 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:23.380733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:23.381475 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:23.381482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:23.381700 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:23.382285 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:23.382312 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:23.384661 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:23.387716 (+ 3055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:23.387725 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:23.387739 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:23.388292 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:23.388604 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:23.388609 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:23.388611 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:23.388835 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:23.389339 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:23.389346 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:23.389732 (+ 386us) client_negotiation.cc:769] Sending connection context
0808 15:07:23.389972 (+ 240us) client_negotiation.cc:240] Negotiation successful
0808 15:07:23.390317 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":264,"threads_started":1}
I20260808 15:07:23.392714 31768 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46269
I20260808 15:07:23.392975 31768 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:23.393646 31768 heartbeater.cc:511] Master 127.2.180.254:46269 requested a full tablet report, sending...
I20260808 15:07:23.395531 31487 ts_manager.cc:194] Registered new tserver with Master: 0d165294cac04ca8b304deaa25641447 (127.2.180.193:36087)
I20260808 15:07:23.396909 31487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.193@KRBTEST.COM'} at 127.2.180.193:53863
WARNING: no policy specified for kudu/127.2.180.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
Entry for principal kudu/127.2.180.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab.
I20260808 15:07:23.434762 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46269
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:23.748267 31776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:23.748509 31776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:23.748607 31776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:23.755460 31776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:23.755553 31776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:23.755662 31776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:07:23.765213 31776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.194.keytab
--principal=kudu/127.2.180.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:23.766489 31776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:23.767688 31776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:23.779529 31782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:23.779599 31783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:23.779567 31785 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:23.780654 31776 server_base.cc:1034] running on GCE node
I20260808 15:07:23.781245 31776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:23.782055 31776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:23.783274 31776 hybrid_clock.cc:648] HybridClock initialized: now 1786201643783214 us; error 43 us; skew 500 ppm
I20260808 15:07:23.783569 31776 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:23 dist-test-slave-jlzn krb5kdc[31428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201643, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:23.786448 31776 init.cc:377] Logged in from keytab as kudu/127.2.180.194@KRBTEST.COM (short username kudu)
I20260808 15:07:23.787770 31776 webserver.cc:460] Webserver started at http://127.2.180.194:46873/ using document root <none> and password file <none>
I20260808 15:07:23.788401 31776 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:23.788517 31776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:23.788801 31776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:23.791530 31776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "396591157692495e82b98342edd09032"
format_stamp: "Formatted at 2026-08-08 15:07:23 on dist-test-slave-jlzn"
server_key: "9fa41c4246e835ff0809765a8f8f486e"
server_key_iv: "af4a5f97903342b9937bde543e2bae52"
server_key_version: "encryptionkey@0"
I20260808 15:07:23.792320 31776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "396591157692495e82b98342edd09032"
format_stamp: "Formatted at 2026-08-08 15:07:23 on dist-test-slave-jlzn"
server_key: "9fa41c4246e835ff0809765a8f8f486e"
server_key_iv: "af4a5f97903342b9937bde543e2bae52"
server_key_version: "encryptionkey@0"
I20260808 15:07:23.798447 31776 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 15:07:23.802628 31794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:23.804052 31776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 15:07:23.804210 31776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "396591157692495e82b98342edd09032"
format_stamp: "Formatted at 2026-08-08 15:07:23 on dist-test-slave-jlzn"
server_key: "9fa41c4246e835ff0809765a8f8f486e"
server_key_iv: "af4a5f97903342b9937bde543e2bae52"
server_key_version: "encryptionkey@0"
I20260808 15:07:23.804400 31776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:23.826864 31776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:23.830539 31776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:23.830757 31776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:23.831593 31776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:23.832825 31776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:23.832901 31776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:23.832986 31776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:23.833027 31776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:23.861858 31776 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:37163
I20260808 15:07:23.861935 31966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:37163 every 8 connection(s)
I20260808 15:07:23.863061 31776 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:07:23.865178 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 31776
I20260808 15:07:23.865365 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:07:23.865891 2771 external_mini_cluster.cc:1428] Setting key b58e36686cc21fd522235c70a5a56244
Aug 08 15:07:23 dist-test-slave-jlzn krb5kdc[31428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201643, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.194@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:23.881588 31770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:23.865486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:40693 (local address 127.2.180.254:46269)
0808 15:07:23.865666 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:23.865670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:23.867030 (+ 1360us) server_negotiation.cc:407] Connection header received
0808 15:07:23.868868 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:23.868874 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:23.868966 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:23.869050 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:23.870447 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:23.871158 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:23.872738 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:23.873143 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:23.873253 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:23.876384 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:23.876388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:23.876392 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:23.876423 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:23.878758 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:23.879416 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:23.879420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:23.879422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:23.879566 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:23.880011 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:23.880014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:23.880016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:23.880267 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:23.880415 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:23.881037 (+ 622us) server_negotiation.cc:299] Negotiation successful
0808 15:07:23.881275 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:07:23.882424 31970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:23.866073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46269 (local address 127.2.180.194:40693)
0808 15:07:23.866877 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:23.866927 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:23.868612 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:23.869285 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:23.869293 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:23.869765 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:23.870282 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:23.870299 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:23.871304 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:23.871308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:23.872592 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:23.872605 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:23.873352 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:23.873359 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:23.873561 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:23.874137 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:23.874167 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:23.876198 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:23.878884 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:23.878893 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:23.878906 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:23.879299 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:23.879670 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:23.879674 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:23.879678 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:23.879901 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:23.880388 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:23.880396 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:23.880691 (+ 295us) client_negotiation.cc:769] Sending connection context
0808 15:07:23.880907 (+ 216us) client_negotiation.cc:240] Negotiation successful
0808 15:07:23.881367 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":312,"threads_started":1}
I20260808 15:07:23.883715 31967 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46269
I20260808 15:07:23.884037 31967 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:23.884810 31967 heartbeater.cc:511] Master 127.2.180.254:46269 requested a full tablet report, sending...
I20260808 15:07:23.886408 31487 ts_manager.cc:194] Registered new tserver with Master: 396591157692495e82b98342edd09032 (127.2.180.194:37163)
I20260808 15:07:23.887449 31487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.194@KRBTEST.COM'} at 127.2.180.194:40693
WARNING: no policy specified for kudu/127.2.180.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
Entry for principal kudu/127.2.180.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab.
I20260808 15:07:23.926011 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46269
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41105
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:24.239061 31973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:24.239301 31973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:24.239391 31973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:24.246230 31973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:24.246322 31973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:24.246433 31973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:07:24.255394 31973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41105
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.195.keytab
--principal=kudu/127.2.180.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:46269
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:24.256486 31973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:24.257658 31973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:24.269549 31983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:24.269690 31982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:24.269656 31985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:24.270511 31973 server_base.cc:1034] running on GCE node
I20260808 15:07:24.271047 31973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:24.271921 31973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:24.273119 31973 hybrid_clock.cc:648] HybridClock initialized: now 1786201644273067 us; error 39 us; skew 500 ppm
I20260808 15:07:24.273401 31973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:24 dist-test-slave-jlzn krb5kdc[31428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201644, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:24.279912 31973 init.cc:377] Logged in from keytab as kudu/127.2.180.195@KRBTEST.COM (short username kudu)
I20260808 15:07:24.281320 31973 webserver.cc:460] Webserver started at http://127.2.180.195:38781/ using document root <none> and password file <none>
I20260808 15:07:24.281955 31973 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:24.282052 31973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:24.282302 31973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:24.285018 31973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "64f27abf18de4a4ab832300bc9a0c8fa"
format_stamp: "Formatted at 2026-08-08 15:07:24 on dist-test-slave-jlzn"
server_key: "d512c1d455dbeb3dc2c8290acd9b65e6"
server_key_iv: "18e02485c1e6c41a409edd8df283d9f5"
server_key_version: "encryptionkey@0"
I20260808 15:07:24.285778 31973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "64f27abf18de4a4ab832300bc9a0c8fa"
format_stamp: "Formatted at 2026-08-08 15:07:24 on dist-test-slave-jlzn"
server_key: "d512c1d455dbeb3dc2c8290acd9b65e6"
server_key_iv: "18e02485c1e6c41a409edd8df283d9f5"
server_key_version: "encryptionkey@0"
I20260808 15:07:24.290407 31973 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:07:24.293561 32002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:24.294721 31973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 15:07:24.294852 31973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "64f27abf18de4a4ab832300bc9a0c8fa"
format_stamp: "Formatted at 2026-08-08 15:07:24 on dist-test-slave-jlzn"
server_key: "d512c1d455dbeb3dc2c8290acd9b65e6"
server_key_iv: "18e02485c1e6c41a409edd8df283d9f5"
server_key_version: "encryptionkey@0"
I20260808 15:07:24.295023 31973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:24.333268 31973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:24.336992 31973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:24.337256 31973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:24.338130 31973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:24.339359 31973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:24.339442 31973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:24.339565 31973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:24.339630 31973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:24.370893 31973 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:38595
I20260808 15:07:24.370949 32167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:38595 every 8 connection(s)
I20260808 15:07:24.372149 31973 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:07:24.377554 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 31973
I20260808 15:07:24.377718 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:07:24.378144 2771 external_mini_cluster.cc:1428] Setting key ff38ebfe7ff1c117e8e20320e7b14fcc
Aug 08 15:07:24 dist-test-slave-jlzn krb5kdc[31428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201644, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.195@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:24.390723 31770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.374584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:60509 (local address 127.2.180.254:46269)
0808 15:07:24.374792 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.374799 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.375784 (+ 985us) server_negotiation.cc:407] Connection header received
0808 15:07:24.377545 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.377551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.377668 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.377765 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:24.379402 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.380378 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.381982 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.382388 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.382478 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.385650 (+ 3172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:24.385654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:24.385658 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:24.385689 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:24.387974 (+ 2285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.388628 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.388634 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.388637 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.388769 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.389167 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.389172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.389174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.389434 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:24.389573 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.390165 (+ 592us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.390380 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 15:07:24.391633 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.374993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:46269 (local address 127.2.180.195:60509)
0808 15:07:24.375626 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:24.375677 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:24.377293 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:24.377993 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:24.378001 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:24.378526 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:24.379224 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.379250 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.380524 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.380530 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.381859 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.381872 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.382546 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.382551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.382738 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.383323 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:07:24.383345 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:24.385452 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:24.388103 (+ 2651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:24.388112 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:24.388126 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:24.388512 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:24.388872 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:07:24.388877 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:07:24.388881 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:07:24.389053 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:07:24.389568 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:24.389575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:24.389838 (+ 263us) client_negotiation.cc:769] Sending connection context
0808 15:07:24.390068 (+ 230us) client_negotiation.cc:240] Negotiation successful
0808 15:07:24.390509 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":255,"threads_started":1}
I20260808 15:07:24.392894 32168 heartbeater.cc:347] Connected to a master server at 127.2.180.254:46269
I20260808 15:07:24.393177 32168 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:24.393803 32168 heartbeater.cc:511] Master 127.2.180.254:46269 requested a full tablet report, sending...
I20260808 15:07:24.395071 31487 ts_manager.cc:194] Registered new tserver with Master: 64f27abf18de4a4ab832300bc9a0c8fa (127.2.180.195:38595)
I20260808 15:07:24.395885 31487 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.2.180.195@KRBTEST.COM'} at 127.2.180.195:60509
I20260808 15:07:24.399430 31768 heartbeater.cc:503] Master 127.2.180.254:46269 was elected leader, sending a full tablet report...
I20260808 15:07:24.404798 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:07:24.419749 31770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.408653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.2.180.254:46269)
0808 15:07:24.408797 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.408801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.409255 (+ 454us) server_negotiation.cc:407] Connection header received
0808 15:07:24.409374 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.409377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.409428 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.409499 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:24.410815 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.411482 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.412691 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.413074 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.413167 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.415223 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:24.415227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:24.415230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:24.415258 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:24.417618 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.418240 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.418244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.418246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.418369 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.418670 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.418673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.418675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.418957 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:24.419134 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.419380 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.419539 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:07:24.423087 31487 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:07:24.429632 31487 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:07:24.450891 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.441601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36360 (local address 127.2.180.194:37163)
0808 15:07:24.442375 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.442383 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.443508 (+ 1125us) server_negotiation.cc:407] Connection header received
0808 15:07:24.443610 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.443617 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.443869 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.444051 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:24.445099 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.446518 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.448878 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.449772 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.449896 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.450095 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.450411 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.450575 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":498,"threads_started":1}
I20260808 15:07:24.450930 32187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.441066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:37163 (local address 127.0.0.1:36360)
0808 15:07:24.443246 (+ 2180us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:24.443259 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:24.443380 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:24.444322 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:24.444326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:24.444355 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:24.444894 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.444905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.446651 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.446656 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.448734 (+ 2078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.448746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.449972 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.449978 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.450203 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.450247 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 15:07:24.450356 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 15:07:24.450523 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2067,"mutex_wait_us":112,"thread_start_us":1895,"threads_started":1}
I20260808 15:07:24.451683 32184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.440676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:38595 (local address 127.0.0.1:36650)
0808 15:07:24.442378 (+ 1702us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:24.442419 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:24.442675 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:24.444144 (+ 1469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:24.444163 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:24.444196 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:24.444618 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.444632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.446385 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.446391 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.448789 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.448805 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.450810 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.450817 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.450989 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.451021 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:07:24.451113 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 15:07:24.451227 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1450,"mutex_wait_us":58,"thread_start_us":1310,"threads_started":1}
I20260808 15:07:24.451733 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.440982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36650 (local address 127.2.180.195:38595)
0808 15:07:24.442008 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.442016 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.442526 (+ 510us) server_negotiation.cc:407] Connection header received
0808 15:07:24.443351 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.443361 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.443603 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.443792 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:24.444993 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.446237 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.448984 (+ 2747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.450332 (+ 1348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.450436 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.450659 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.451194 (+ 535us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.451356 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":855,"thread_start_us":699,"threads_started":1}
I20260808 15:07:24.453749 32190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.443139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:36087 (local address 127.0.0.1:56260)
0808 15:07:24.444705 (+ 1566us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:24.444723 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:24.444835 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:24.445918 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:24.445923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:24.445948 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:24.446257 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.446268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.448105 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.448110 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.450245 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.450262 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.452988 (+ 2726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.452995 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.453169 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.453199 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 15:07:24.453293 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 15:07:24.453410 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1408,"mutex_wait_us":5,"thread_start_us":1633,"threads_started":1}
I20260808 15:07:24.453797 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.443466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56260 (local address 127.2.180.193:36087)
0808 15:07:24.445378 (+ 1912us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.445386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.445411 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:07:24.445502 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.445509 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.445726 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.445899 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:24.446420 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.447912 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.450436 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.452787 (+ 2351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.452925 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.453147 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.453348 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.453504 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1745,"thread_start_us":1602,"threads_started":1}
I20260808 15:07:24.454962 31875 tablet_service.cc:1467] Processing CreateTablet for tablet 7997ae422859486f81ecb6b00a15b937 (DEFAULT_TABLE table=test-table [id=000d9c66b62543a5b922e03595e648ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:24.455025 32077 tablet_service.cc:1467] Processing CreateTablet for tablet 7997ae422859486f81ecb6b00a15b937 (DEFAULT_TABLE table=test-table [id=000d9c66b62543a5b922e03595e648ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:24.456128 32077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7997ae422859486f81ecb6b00a15b937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:24.456175 31875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7997ae422859486f81ecb6b00a15b937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:24.459725 31665 tablet_service.cc:1467] Processing CreateTablet for tablet 7997ae422859486f81ecb6b00a15b937 (DEFAULT_TABLE table=test-table [id=000d9c66b62543a5b922e03595e648ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:24.461030 31665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7997ae422859486f81ecb6b00a15b937. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:24.466997 32193 tablet_bootstrap.cc:492] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Bootstrap starting.
I20260808 15:07:24.469892 32194 tablet_bootstrap.cc:492] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Bootstrap starting.
I20260808 15:07:24.471143 32193 tablet_bootstrap.cc:654] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:24.472410 32193 log.cc:824] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:24.473307 32194 tablet_bootstrap.cc:654] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:24.473364 32196 tablet_bootstrap.cc:492] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Bootstrap starting.
I20260808 15:07:24.474613 32194 log.cc:824] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:24.476260 32196 tablet_bootstrap.cc:654] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:24.477254 32194 tablet_bootstrap.cc:492] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: No bootstrap required, opened a new log
I20260808 15:07:24.477415 32196 log.cc:824] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:24.477480 32194 ts_tablet_manager.cc:1397] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260808 15:07:24.477643 32193 tablet_bootstrap.cc:492] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: No bootstrap required, opened a new log
I20260808 15:07:24.477892 32193 ts_tablet_manager.cc:1397] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260808 15:07:24.480249 32196 tablet_bootstrap.cc:492] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: No bootstrap required, opened a new log
I20260808 15:07:24.480496 32196 ts_tablet_manager.cc:1397] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260808 15:07:24.484783 32193 raft_consensus.cc:348] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.485092 32193 raft_consensus.cc:374] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:24.485193 32193 raft_consensus.cc:729] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396591157692495e82b98342edd09032, State: Initialized, Role: FOLLOWER
I20260808 15:07:24.485720 32193 consensus_queue.cc:260] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.486665 31967 heartbeater.cc:503] Master 127.2.180.254:46269 was elected leader, sending a full tablet report...
I20260808 15:07:24.487309 32193 ts_tablet_manager.cc:1428] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260808 15:07:24.487669 32194 raft_consensus.cc:348] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.488086 32194 raft_consensus.cc:374] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:24.488183 32194 raft_consensus.cc:729] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64f27abf18de4a4ab832300bc9a0c8fa, State: Initialized, Role: FOLLOWER
I20260808 15:07:24.488727 32194 consensus_queue.cc:260] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.489746 32168 heartbeater.cc:503] Master 127.2.180.254:46269 was elected leader, sending a full tablet report...
I20260808 15:07:24.490406 32194 ts_tablet_manager.cc:1428] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
I20260808 15:07:24.490265 32196 raft_consensus.cc:348] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.490667 32196 raft_consensus.cc:374] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:24.490801 32196 raft_consensus.cc:729] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d165294cac04ca8b304deaa25641447, State: Initialized, Role: FOLLOWER
I20260808 15:07:24.491489 32196 consensus_queue.cc:260] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.493212 32196 ts_tablet_manager.cc:1428] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260808 15:07:24.579860 32208 raft_consensus.cc:482] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:07:24.580138 32208 raft_consensus.cc:504] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.581740 32208 leader_election.cc:288] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 64f27abf18de4a4ab832300bc9a0c8fa (127.2.180.195:38595), 396591157692495e82b98342edd09032 (127.2.180.194:37163)
I20260808 15:07:24.588056 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.582010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:35977 (local address 127.2.180.195:38595)
0808 15:07:24.582214 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.582221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.582872 (+ 651us) server_negotiation.cc:407] Connection header received
0808 15:07:24.583053 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.583056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.583117 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.583196 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:24.583842 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.584561 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.586396 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.587147 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.587299 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.587370 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.587664 (+ 294us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.587784 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 15:07:24.588083 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.582272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:46187 (local address 127.2.180.194:37163)
0808 15:07:24.582465 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.582471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.583050 (+ 579us) server_negotiation.cc:407] Connection header received
0808 15:07:24.583217 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.583221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.583278 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.583348 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:24.583991 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.584686 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.586388 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.587153 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.587285 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.587358 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.587694 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.587844 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:07:24.588222 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.581915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:38595 (local address 127.2.180.193:35977)
0808 15:07:24.582760 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:24.582776 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:24.582916 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:24.583223 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:24.583228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:24.583252 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:24.583670 (+ 418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.583680 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.584692 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.584696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.586243 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.586256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.587369 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.587375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.587513 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.587538 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 15:07:24.587618 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 15:07:24.587748 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"mutex_wait_us":106,"thread_start_us":499,"threads_started":1}
I20260808 15:07:24.588440 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.582199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:37163 (local address 127.2.180.193:46187)
0808 15:07:24.582949 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:24.582963 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:24.583095 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:24.583390 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:24.583395 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:24.583423 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:24.583834 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.583846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.584825 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.584829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.586242 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:24.586255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.587428 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.587433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:24.587547 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.587572 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 15:07:24.587651 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 15:07:24.587767 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":23,"thread_start_us":439,"threads_started":1}
I20260808 15:07:24.589097 32104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7997ae422859486f81ecb6b00a15b937" candidate_uuid: "0d165294cac04ca8b304deaa25641447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" is_pre_election: true
I20260808 15:07:24.589262 31910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7997ae422859486f81ecb6b00a15b937" candidate_uuid: "0d165294cac04ca8b304deaa25641447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396591157692495e82b98342edd09032" is_pre_election: true
I20260808 15:07:24.589577 32104 raft_consensus.cc:2454] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d165294cac04ca8b304deaa25641447 in term 0.
I20260808 15:07:24.589716 31910 raft_consensus.cc:2454] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0d165294cac04ca8b304deaa25641447 in term 0.
I20260808 15:07:24.590201 31607 leader_election.cc:302] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d165294cac04ca8b304deaa25641447, 64f27abf18de4a4ab832300bc9a0c8fa; no voters:
I20260808 15:07:24.590595 32208 raft_consensus.cc:2781] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:07:24.590734 32208 raft_consensus.cc:482] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:07:24.590827 32208 raft_consensus.cc:3037] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:24.593272 32208 raft_consensus.cc:504] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.593796 32208 leader_election.cc:288] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [CANDIDATE]: Term 1 election: Requested vote from peers 64f27abf18de4a4ab832300bc9a0c8fa (127.2.180.195:38595), 396591157692495e82b98342edd09032 (127.2.180.194:37163)
I20260808 15:07:24.594233 31910 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7997ae422859486f81ecb6b00a15b937" candidate_uuid: "0d165294cac04ca8b304deaa25641447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396591157692495e82b98342edd09032"
I20260808 15:07:24.594233 32104 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7997ae422859486f81ecb6b00a15b937" candidate_uuid: "0d165294cac04ca8b304deaa25641447" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64f27abf18de4a4ab832300bc9a0c8fa"
I20260808 15:07:24.594389 32104 raft_consensus.cc:3037] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:24.594403 31910 raft_consensus.cc:3037] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:24.596585 31910 raft_consensus.cc:2454] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d165294cac04ca8b304deaa25641447 in term 1.
I20260808 15:07:24.596609 32104 raft_consensus.cc:2454] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0d165294cac04ca8b304deaa25641447 in term 1.
I20260808 15:07:24.596962 31607 leader_election.cc:302] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0d165294cac04ca8b304deaa25641447, 396591157692495e82b98342edd09032, 64f27abf18de4a4ab832300bc9a0c8fa; no voters:
I20260808 15:07:24.597273 32208 raft_consensus.cc:2781] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:24.597692 32208 raft_consensus.cc:686] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 1 LEADER]: Becoming Leader. State: Replica: 0d165294cac04ca8b304deaa25641447, State: Running, Role: LEADER
I20260808 15:07:24.598181 32208 consensus_queue.cc:237] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } }
I20260808 15:07:24.602674 31485 catalog_manager.cc:5495] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0d165294cac04ca8b304deaa25641447 (127.2.180.193). New cstate: current_term: 1 leader_uuid: "0d165294cac04ca8b304deaa25641447" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0d165294cac04ca8b304deaa25641447" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 36087 } health_report { overall_health: HEALTHY } } }
W20260808 15:07:24.614845 31969 tablet.cc:2369] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:07:24.623627 31769 tablet.cc:2369] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 15:07:24.623790 32169 tablet.cc:2369] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:07:24.639453 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.634844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56272 (local address 127.2.180.193:36087)
0808 15:07:24.634994 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.634999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.635019 (+ 20us) server_negotiation.cc:407] Connection header received
0808 15:07:24.635201 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.635206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.635292 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.635437 (+ 145us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:07:24.635696 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.636470 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.637763 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.638145 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.638232 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.638596 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:07:24.639070 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:07:24.639174 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.639210 (+ 36us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.639272 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 15:07:24.646734 32104 raft_consensus.cc:1261] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 1 FOLLOWER]: Refusing update from remote peer 0d165294cac04ca8b304deaa25641447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:24.646733 31910 raft_consensus.cc:1261] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 1 FOLLOWER]: Refusing update from remote peer 0d165294cac04ca8b304deaa25641447: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:24.647584 32212 consensus_queue.cc:1035] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64f27abf18de4a4ab832300bc9a0c8fa" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 38595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:24.647883 32208 consensus_queue.cc:1035] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396591157692495e82b98342edd09032" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 37163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:24.658457 32218 mvcc.cc:205] Tried to move back new op lower bound from 7316281936464609280 to 7316281936277696512. Current Snapshot: MvccSnapshot[applied={T|T < 7316281936464609280}]
I20260808 15:07:24.658691 32220 mvcc.cc:205] Tried to move back new op lower bound from 7316281936464609280 to 7316281936277696512. Current Snapshot: MvccSnapshot[applied={T|T < 7316281936464609280}]
I20260808 15:07:24.671681 31485 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:24.671988 31485 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:24.675402 31485 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4cf7fbf1c9c4ee890ce74f391f7cfee: Sending DeleteTablet for 3 replicas of tablet 7997ae422859486f81ecb6b00a15b937
I20260808 15:07:24.676399 32077 tablet_service.cc:1514] Processing DeleteTablet for tablet 7997ae422859486f81ecb6b00a15b937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:24 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:36650
I20260808 15:07:24.676463 31875 tablet_service.cc:1514] Processing DeleteTablet for tablet 7997ae422859486f81ecb6b00a15b937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:24 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:36360
I20260808 15:07:24.677377 31665 tablet_service.cc:1514] Processing DeleteTablet for tablet 7997ae422859486f81ecb6b00a15b937 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:24 UTC) from {username='kudu', principal='kudu/127.2.180.254@KRBTEST.COM'} at 127.0.0.1:56260
I20260808 15:07:24.677549 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 31575
I20260808 15:07:24.690017 32235 tablet_replica.cc:330] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: stopping tablet replica
I20260808 15:07:24.690411 32235 raft_consensus.cc:2229] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:07:24.691027 32235 raft_consensus.cc:2258] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:24.692835 32233 tablet_replica.cc:330] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: stopping tablet replica
I20260808 15:07:24.693216 32233 raft_consensus.cc:2229] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:24.693367 32235 ts_tablet_manager.cc:1905] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:24.693686 32233 raft_consensus.cc:2258] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:24.699117 32232 tablet_replica.cc:330] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: stopping tablet replica
I20260808 15:07:24.699440 32232 raft_consensus.cc:2229] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:24.700021 32232 raft_consensus.cc:2258] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:24.700279 32233 ts_tablet_manager.cc:1905] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:24.701663 32235 ts_tablet_manager.cc:1918] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:24.701843 32235 log.cc:1196] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/7997ae422859486f81ecb6b00a15b937
I20260808 15:07:24.702394 32235 ts_tablet_manager.cc:1939] T 7997ae422859486f81ecb6b00a15b937 P 0d165294cac04ca8b304deaa25641447: Deleting consensus metadata
Aug 08 15:07:24 dist-test-slave-jlzn krb5kdc[31428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.193@KRBTEST.COM
I20260808 15:07:24.704113 31465 catalog_manager.cc:4841] TS 0d165294cac04ca8b304deaa25641447 (127.2.180.193:36087): tablet 7997ae422859486f81ecb6b00a15b937 (table test-table [id=000d9c66b62543a5b922e03595e648ed]) successfully deleted
I20260808 15:07:24.706396 32233 ts_tablet_manager.cc:1918] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:24.706542 32233 log.cc:1196] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/7997ae422859486f81ecb6b00a15b937
I20260808 15:07:24.706952 32233 ts_tablet_manager.cc:1939] T 7997ae422859486f81ecb6b00a15b937 P 64f27abf18de4a4ab832300bc9a0c8fa: Deleting consensus metadata
I20260808 15:07:24.708307 31465 catalog_manager.cc:4841] TS 64f27abf18de4a4ab832300bc9a0c8fa (127.2.180.195:38595): tablet 7997ae422859486f81ecb6b00a15b937 (table test-table [id=000d9c66b62543a5b922e03595e648ed]) successfully deleted
I20260808 15:07:24.709312 32232 ts_tablet_manager.cc:1905] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:24.712779 32191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.678989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56278 (local address 127.2.180.193:36087)
0808 15:07:24.679156 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.679163 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.690199 (+ 11036us) server_negotiation.cc:407] Connection header received
0808 15:07:24.690459 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.690464 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.690544 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.690617 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:24.694085 (+ 3468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.695218 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.697864 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.698581 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.698683 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.704988 (+ 6305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:24.705011 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:24.705027 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:24.705079 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:24.709177 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.710117 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.710126 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.710130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.710300 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.710766 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.710771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.710775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.711214 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:24.711525 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.712179 (+ 654us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.712414 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:07:24.714073 32232 ts_tablet_manager.cc:1918] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:24.714176 31762 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:24.714232 32232 log.cc:1196] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/7997ae422859486f81ecb6b00a15b937
I20260808 15:07:24.714757 32232 ts_tablet_manager.cc:1939] T 7997ae422859486f81ecb6b00a15b937 P 396591157692495e82b98342edd09032: Deleting consensus metadata
I20260808 15:07:24.716419 31467 catalog_manager.cc:4841] TS 396591157692495e82b98342edd09032 (127.2.180.194:37163): tablet 7997ae422859486f81ecb6b00a15b937 (table test-table [id=000d9c66b62543a5b922e03595e648ed]) successfully deleted
I20260808 15:07:24.809252 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 31575
I20260808 15:07:24.833854 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 31776
Aug 08 15:07:24 dist-test-slave-jlzn krb5kdc[31428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.194@KRBTEST.COM
I20260808 15:07:24.847743 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.834548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36368 (local address 127.2.180.194:37163)
0808 15:07:24.834728 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.834733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.834808 (+ 75us) server_negotiation.cc:407] Connection header received
0808 15:07:24.834995 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.834998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.835068 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.835180 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:24.836636 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.837355 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.838556 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.838935 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.839023 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.842969 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:24.842982 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:24.842997 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:24.843032 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:24.845584 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.846146 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.846152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.846154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.846267 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.846568 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.846570 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.846572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.846928 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:24.847152 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.847348 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.847499 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:07:24.848412 31961 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:24.935103 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 31776
I20260808 15:07:24.957239 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 31973
Aug 08 15:07:24 dist-test-slave-jlzn krb5kdc[31428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201642, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.195@KRBTEST.COM
I20260808 15:07:24.971118 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:24.957960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.2.180.195:38595)
0808 15:07:24.958128 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:24.958133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:24.958152 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:07:24.958270 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:24.958274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:24.958345 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:24.958427 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:24.959677 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.960442 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.961736 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:24.962176 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:24.962274 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:24.966362 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:24.966378 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:24.966395 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:24.966426 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:24.968908 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.969503 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.969509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.969511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.969651 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:24.969953 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:24.969955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:24.969957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:24.970287 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:24.970496 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:24.970724 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 15:07:24.970877 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:07:24.971714 32163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:25.057353 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 31973
I20260808 15:07:25.078433 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 31441
I20260808 15:07:25.079492 31532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:25.186347 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 31441
2026-08-08T15:07:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4234 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T15:07:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:25Z Disabled control of system clock
I20260808 15:07:25.266389 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:44035
--webserver_interface=127.2.180.254
--webserver_port=0
--builtin_ntp_servers=127.2.180.212:41555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:44035
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:07:25.585022 32245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:25.585278 32245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:25.585373 32245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:25.592415 32245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:25.592527 32245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:25.592572 32245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:25.592620 32245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:25.592664 32245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:25.601722 32245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:44035
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:44035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:25.602891 32245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:25.604596 32245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:25.615994 32250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:25.615965 32257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:25.615926 32254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:25.616683 32245 server_base.cc:1034] running on GCE node
I20260808 15:07:25.617390 32245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:25.618635 32245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:25.619856 32245 hybrid_clock.cc:648] HybridClock initialized: now 1786201645619812 us; error 32 us; skew 500 ppm
I20260808 15:07:25.620183 32245 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:25.622366 32245 webserver.cc:460] Webserver started at http://127.2.180.254:33013/ using document root <none> and password file <none>
I20260808 15:07:25.622978 32245 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:25.623072 32245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:25.623384 32245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:25.626085 32245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "6a0a83afd3be47d2adc310893a9730f6"
format_stamp: "Formatted at 2026-08-08 15:07:25 on dist-test-slave-jlzn"
server_key: "f7ccc3ddc613af4fc9055835f1eead75"
server_key_iv: "8ea0635e62ee7ec641dce0553aeb2555"
server_key_version: "encryptionkey@0"
I20260808 15:07:25.626849 32245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "6a0a83afd3be47d2adc310893a9730f6"
format_stamp: "Formatted at 2026-08-08 15:07:25 on dist-test-slave-jlzn"
server_key: "f7ccc3ddc613af4fc9055835f1eead75"
server_key_iv: "8ea0635e62ee7ec641dce0553aeb2555"
server_key_version: "encryptionkey@0"
I20260808 15:07:25.631593 32245 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 15:07:25.634752 32266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:25.636039 32245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 15:07:25.636209 32245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "6a0a83afd3be47d2adc310893a9730f6"
format_stamp: "Formatted at 2026-08-08 15:07:25 on dist-test-slave-jlzn"
server_key: "f7ccc3ddc613af4fc9055835f1eead75"
server_key_iv: "8ea0635e62ee7ec641dce0553aeb2555"
server_key_version: "encryptionkey@0"
I20260808 15:07:25.636392 32245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:25.664052 32245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:25.664767 32245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:25.665017 32245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:25.681483 32245 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:44035
I20260808 15:07:25.681529 32352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:44035 every 8 connection(s)
I20260808 15:07:25.682660 32245 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:25.686091 32354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:25.686385 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 32245
I20260808 15:07:25.686553 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:25.686982 2771 external_mini_cluster.cc:1428] Setting key dde6e9f7ec398565e32f721fdbc4875f
I20260808 15:07:25.699764 32354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6: Bootstrap starting.
I20260808 15:07:25.701752 32358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:25.688797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.2.180.254:44035)
0808 15:07:25.689811 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:25.689824 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:25.689872 (+ 48us) server_negotiation.cc:407] Connection header received
0808 15:07:25.692334 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:25.692361 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:25.692932 (+ 571us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:25.693358 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:25.693842 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:25.695672 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:25.696958 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:25.697994 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:25.698156 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:25.699119 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:25.699129 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:25.699145 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:25.699825 (+ 680us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:25.699925 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:25.699952 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:25.700059 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:25.700249 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 15:07:25.700544 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":512,"thread_start_us":312,"threads_started":1}
I20260808 15:07:25.703404 32354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:25.704643 32354 log.cc:824] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:25.707288 32354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6: No bootstrap required, opened a new log
I20260808 15:07:25.714262 32354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0a83afd3be47d2adc310893a9730f6" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44035 } }
I20260808 15:07:25.714551 32354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:25.714619 32354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a0a83afd3be47d2adc310893a9730f6, State: Initialized, Role: FOLLOWER
I20260808 15:07:25.715049 32354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0a83afd3be47d2adc310893a9730f6" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44035 } }
I20260808 15:07:25.715255 32354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:25.715345 32354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:25.715463 32354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:25.717433 32354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0a83afd3be47d2adc310893a9730f6" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44035 } }
I20260808 15:07:25.717839 32354 leader_election.cc:302] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a0a83afd3be47d2adc310893a9730f6; no voters:
I20260808 15:07:25.718398 32354 leader_election.cc:288] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:25.718492 32360 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:25.719302 32360 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [term 1 LEADER]: Becoming Leader. State: Replica: 6a0a83afd3be47d2adc310893a9730f6, State: Running, Role: LEADER
I20260808 15:07:25.719772 32360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0a83afd3be47d2adc310893a9730f6" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44035 } }
I20260808 15:07:25.720285 32354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:25.722358 32361 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a0a83afd3be47d2adc310893a9730f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0a83afd3be47d2adc310893a9730f6" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44035 } } }
I20260808 15:07:25.722805 32361 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:25.722898 32362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a0a83afd3be47d2adc310893a9730f6. Latest consensus state: current_term: 1 leader_uuid: "6a0a83afd3be47d2adc310893a9730f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a0a83afd3be47d2adc310893a9730f6" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44035 } } }
I20260808 15:07:25.723186 32362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:25.724963 32373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:25.734977 32373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:25.742944 32373 catalog_manager.cc:1347] Generated new cluster ID: 9fcfe49f761a48fb8886cfa0f7577c2f
I20260808 15:07:25.743084 32373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:25.759487 32373 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:25.760639 32373 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:25.766876 32373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6: Generated new TSK 0
I20260808 15:07:25.767518 32373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:25.781234 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.193:0
--local_ip_for_outbound_sockets=127.2.180.193
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44035
--builtin_ntp_servers=127.2.180.212:41555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 15:07:26.107290 32385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:26.107539 32385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:26.107618 32385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:26.114431 32385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:26.114516 32385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:26.114610 32385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.193
I20260808 15:07:26.123200 32385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.2.180.193
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.193
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:26.124308 32385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:26.126027 32385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:26.137946 32392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:26.138067 32395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:26.138100 32398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:26.139211 32385 server_base.cc:1034] running on GCE node
I20260808 15:07:26.139786 32385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:26.140763 32385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:26.141955 32385 hybrid_clock.cc:648] HybridClock initialized: now 1786201646141911 us; error 33 us; skew 500 ppm
I20260808 15:07:26.142239 32385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:26.144241 32385 webserver.cc:460] Webserver started at http://127.2.180.193:43361/ using document root <none> and password file <none>
I20260808 15:07:26.144863 32385 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:26.144956 32385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:26.145263 32385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:26.147836 32385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/data/instance:
uuid: "8adc7f5c772948c8887274592cfe9617"
format_stamp: "Formatted at 2026-08-08 15:07:26 on dist-test-slave-jlzn"
server_key: "0d8c86e1bde74089047957cf16d19861"
server_key_iv: "b1653c8282fe305dc7f9c1d9e162a7af"
server_key_version: "encryptionkey@0"
I20260808 15:07:26.148582 32385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance:
uuid: "8adc7f5c772948c8887274592cfe9617"
format_stamp: "Formatted at 2026-08-08 15:07:26 on dist-test-slave-jlzn"
server_key: "0d8c86e1bde74089047957cf16d19861"
server_key_iv: "b1653c8282fe305dc7f9c1d9e162a7af"
server_key_version: "encryptionkey@0"
I20260808 15:07:26.153219 32385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 15:07:26.156350 32405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:26.157529 32385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:07:26.157675 32385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/wal
uuid: "8adc7f5c772948c8887274592cfe9617"
format_stamp: "Formatted at 2026-08-08 15:07:26 on dist-test-slave-jlzn"
server_key: "0d8c86e1bde74089047957cf16d19861"
server_key_iv: "b1653c8282fe305dc7f9c1d9e162a7af"
server_key_version: "encryptionkey@0"
I20260808 15:07:26.157863 32385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:26.182090 32385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:26.182792 32385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:26.183068 32385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:26.183946 32385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:26.185271 32385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:26.185366 32385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:26.185468 32385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:26.185541 32385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:26.214687 32385 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.193:43625
I20260808 15:07:26.214751 32570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.193:43625 every 8 connection(s)
I20260808 15:07:26.215917 32385 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/data/info.pb
I20260808 15:07:26.223018 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 32385
I20260808 15:07:26.223166 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/wal/instance
I20260808 15:07:26.223594 2771 external_mini_cluster.cc:1428] Setting key 27a6accb97cd6aa32e537de53cfbb24b
I20260808 15:07:26.226492 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.194:0
--local_ip_for_outbound_sockets=127.2.180.194
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44035
--builtin_ntp_servers=127.2.180.212:41555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:07:26.228076 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:26.218203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.193:34351 (local address 127.2.180.254:44035)
0808 15:07:26.218612 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:26.218616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:26.219594 (+ 978us) server_negotiation.cc:407] Connection header received
0808 15:07:26.221442 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:26.221447 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:26.221516 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:26.221643 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:26.222643 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:26.223445 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:26.225426 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:26.225882 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:26.225973 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:26.227379 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:26.227384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:26.227387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:26.227403 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:26.227440 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:26.227452 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:26.227516 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:26.227732 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 15:07:26.227830 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":217,"threads_started":1}
I20260808 15:07:26.229211 32574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:26.218630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44035 (local address 127.2.180.193:34351)
0808 15:07:26.219449 (+ 819us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:26.219490 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:26.221174 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:26.221869 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:26.221881 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:26.221967 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:26.222485 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:26.222501 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:26.223586 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:26.223592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:26.225317 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:26.225329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:26.226099 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:26.226107 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:26.226388 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:26.227057 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:26.227061 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:26.227106 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:26.227198 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:26.227533 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:26.227542 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:26.227548 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 15:07:26.227720 (+ 172us) client_negotiation.cc:240] Negotiation successful
0808 15:07:26.228099 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":242,"threads_started":1}
I20260808 15:07:26.230541 32571 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44035
I20260808 15:07:26.230803 32571 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:26.231473 32571 heartbeater.cc:511] Master 127.2.180.254:44035 requested a full tablet report, sending...
I20260808 15:07:26.233323 32290 ts_manager.cc:194] Registered new tserver with Master: 8adc7f5c772948c8887274592cfe9617 (127.2.180.193:43625)
I20260808 15:07:26.234728 32290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.193:34351
W20260808 15:07:26.542191 32575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:26.542470 32575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:26.542604 32575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:26.549431 32575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:26.549516 32575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:26.549666 32575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.194
I20260808 15:07:26.558130 32575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.2.180.194
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.194
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:26.559173 32575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:26.560837 32575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:26.572880 32582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:26.572844 32583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:26.572947 32586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:26.573807 32575 server_base.cc:1034] running on GCE node
I20260808 15:07:26.574319 32575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:26.575105 32575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:26.576288 32575 hybrid_clock.cc:648] HybridClock initialized: now 1786201646576239 us; error 42 us; skew 500 ppm
I20260808 15:07:26.576561 32575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:26.578536 32575 webserver.cc:460] Webserver started at http://127.2.180.194:38545/ using document root <none> and password file <none>
I20260808 15:07:26.579126 32575 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:26.579228 32575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:26.579526 32575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:26.582144 32575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/data/instance:
uuid: "23d1dccb22ba484fb44d27e0a546e277"
format_stamp: "Formatted at 2026-08-08 15:07:26 on dist-test-slave-jlzn"
server_key: "d81cf36338544e07a1ab37da9b8e1fdc"
server_key_iv: "835d9747d73c649d8d3255139d56e68e"
server_key_version: "encryptionkey@0"
I20260808 15:07:26.582880 32575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance:
uuid: "23d1dccb22ba484fb44d27e0a546e277"
format_stamp: "Formatted at 2026-08-08 15:07:26 on dist-test-slave-jlzn"
server_key: "d81cf36338544e07a1ab37da9b8e1fdc"
server_key_iv: "835d9747d73c649d8d3255139d56e68e"
server_key_version: "encryptionkey@0"
I20260808 15:07:26.587399 32575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 15:07:26.590509 32595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:26.591655 32575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:07:26.591799 32575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/wal
uuid: "23d1dccb22ba484fb44d27e0a546e277"
format_stamp: "Formatted at 2026-08-08 15:07:26 on dist-test-slave-jlzn"
server_key: "d81cf36338544e07a1ab37da9b8e1fdc"
server_key_iv: "835d9747d73c649d8d3255139d56e68e"
server_key_version: "encryptionkey@0"
I20260808 15:07:26.592021 32575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:26.620903 32575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:26.621740 32575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:26.622005 32575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:26.622944 32575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:26.624456 32575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:26.624552 32575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:26.624667 32575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:26.624742 32575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:26.652963 32575 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.194:42445
I20260808 15:07:26.653034 32759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.194:42445 every 8 connection(s)
I20260808 15:07:26.654124 32575 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/data/info.pb
I20260808 15:07:26.664130 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 32575
I20260808 15:07:26.664271 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/wal/instance
I20260808 15:07:26.664669 2771 external_mini_cluster.cc:1428] Setting key f236d949127e642d8b811df0b1a435f6
I20260808 15:07:26.665380 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:26.656360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.194:59689 (local address 127.2.180.254:44035)
0808 15:07:26.656527 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:26.656531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:26.657784 (+ 1253us) server_negotiation.cc:407] Connection header received
0808 15:07:26.659419 (+ 1635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:26.659423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:26.659490 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:26.659563 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:26.660541 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:26.661255 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:26.662855 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:26.663244 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:26.663335 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:26.664581 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:26.664586 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:26.664590 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:26.664608 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:26.664639 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:26.664651 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:26.664729 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:26.664949 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 15:07:26.665053 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:07:26.666334 32765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:26.656821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44035 (local address 127.2.180.194:59689)
0808 15:07:26.657644 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:26.657678 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:26.659184 (+ 1506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:26.659743 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:26.659750 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:26.659845 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:26.660374 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:26.660390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:26.661391 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:26.661395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:26.662729 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:26.662742 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:26.663402 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:26.663406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:26.663613 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:26.664270 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:26.664274 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:26.664316 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:26.664401 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:26.664741 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:26.664752 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:26.664761 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 15:07:26.664917 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 15:07:26.665249 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":257,"threads_started":1}
I20260808 15:07:26.667272 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.2.180.195:0
--local_ip_for_outbound_sockets=127.2.180.195
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44035
--builtin_ntp_servers=127.2.180.212:41555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 15:07:26.667541 32760 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44035
I20260808 15:07:26.667794 32760 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:26.668486 32760 heartbeater.cc:511] Master 127.2.180.254:44035 requested a full tablet report, sending...
I20260808 15:07:26.669801 32290 ts_manager.cc:194] Registered new tserver with Master: 23d1dccb22ba484fb44d27e0a546e277 (127.2.180.194:42445)
I20260808 15:07:26.670728 32290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.194:59689
W20260808 15:07:26.984894 300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:26.985116 300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:26.985193 300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:26.991942 300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:26.992024 300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:26.992125 300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.2.180.195
I20260808 15:07:27.000559 300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.2.180.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.2.180.195
--webserver_port=0
--tserver_master_addrs=127.2.180.254:44035
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.2.180.195
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:27.001565 300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:27.003208 300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:27.015067 310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:27.015045 311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:27.015147 313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:27.015690 300 server_base.cc:1034] running on GCE node
I20260808 15:07:27.016266 300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:27.017066 300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:27.018263 300 hybrid_clock.cc:648] HybridClock initialized: now 1786201647018220 us; error 31 us; skew 500 ppm
I20260808 15:07:27.018573 300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 15:07:27.020704 300 webserver.cc:460] Webserver started at http://127.2.180.195:40717/ using document root <none> and password file <none>
I20260808 15:07:27.021363 300 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:27.021457 300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:27.021764 300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:27.024573 300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/data/instance:
uuid: "17a11551a5614288afcce7d86c01db1b"
format_stamp: "Formatted at 2026-08-08 15:07:27 on dist-test-slave-jlzn"
server_key: "dbf6f898a94126e3d07607f4e201c829"
server_key_iv: "96cdbdfe2ccbcb750bf7dc5d9e44355b"
server_key_version: "encryptionkey@0"
I20260808 15:07:27.025362 300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance:
uuid: "17a11551a5614288afcce7d86c01db1b"
format_stamp: "Formatted at 2026-08-08 15:07:27 on dist-test-slave-jlzn"
server_key: "dbf6f898a94126e3d07607f4e201c829"
server_key_iv: "96cdbdfe2ccbcb750bf7dc5d9e44355b"
server_key_version: "encryptionkey@0"
I20260808 15:07:27.030016 300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:07:27.033275 321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:27.034405 300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:07:27.034584 300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/wal
uuid: "17a11551a5614288afcce7d86c01db1b"
format_stamp: "Formatted at 2026-08-08 15:07:27 on dist-test-slave-jlzn"
server_key: "dbf6f898a94126e3d07607f4e201c829"
server_key_iv: "96cdbdfe2ccbcb750bf7dc5d9e44355b"
server_key_version: "encryptionkey@0"
I20260808 15:07:27.034775 300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:27.072333 300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:27.073036 300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:27.073294 300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:27.074149 300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 15:07:27.075460 300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 15:07:27.075556 300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:27.075661 300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 15:07:27.075729 300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:27.103543 300 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.195:41231
I20260808 15:07:27.103602 496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.195:41231 every 8 connection(s)
I20260808 15:07:27.104732 300 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/data/info.pb
I20260808 15:07:27.114341 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 300
I20260808 15:07:27.114497 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/wal/instance
I20260808 15:07:27.114964 2771 external_mini_cluster.cc:1428] Setting key f1dcd2b2836b0cc9fa5c2ddec82be203
I20260808 15:07:27.116359 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.107051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:54841 (local address 127.2.180.254:44035)
0808 15:07:27.107217 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.107222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.108494 (+ 1272us) server_negotiation.cc:407] Connection header received
0808 15:07:27.110214 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.110219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.110296 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.110383 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:27.111379 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.112162 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.113662 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.114102 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.114216 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.115530 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:27.115550 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:27.115554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:27.115571 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:27.115604 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:27.115616 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:27.115689 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.115965 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.116070 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:07:27.117291 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.107473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:44035 (local address 127.2.180.195:54841)
0808 15:07:27.108341 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.108386 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:27.109950 (+ 1564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:27.110607 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:27.110615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:27.110700 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:07:27.111217 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.111234 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.112323 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.112326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.113520 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.113533 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.114314 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.114321 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.114550 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.115226 (+ 676us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 15:07:27.115228 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:07:27.115264 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 15:07:27.115342 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:07:27.115716 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:07:27.115726 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:07:27.115734 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 15:07:27.115929 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 15:07:27.116280 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":280,"threads_started":1}
I20260808 15:07:27.118468 497 heartbeater.cc:347] Connected to a master server at 127.2.180.254:44035
I20260808 15:07:27.118731 497 heartbeater.cc:464] Registering TS with master...
I20260808 15:07:27.119401 497 heartbeater.cc:511] Master 127.2.180.254:44035 requested a full tablet report, sending...
I20260808 15:07:27.120738 32290 ts_manager.cc:194] Registered new tserver with Master: 17a11551a5614288afcce7d86c01db1b (127.2.180.195:41231)
I20260808 15:07:27.121542 32290 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.2.180.195:54841
I20260808 15:07:27.129580 2771 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 15:07:27.138912 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.133408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37108 (local address 127.2.180.254:44035)
0808 15:07:27.133589 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.133596 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.133929 (+ 333us) server_negotiation.cc:407] Connection header received
0808 15:07:27.134079 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.134084 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.134166 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.134261 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:27.134656 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.135297 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.136676 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.137055 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.137140 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.138247 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:27.138254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:27.138258 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:27.138279 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:27.138322 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:27.138336 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:27.138404 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.138524 (+ 120us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.138630 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:07:27.142257 32290 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 15:07:27.148720 32290 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 15:07:27.180141 514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.159451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.195:41231 (local address 127.0.0.1:55444)
0808 15:07:27.171913 (+ 12462us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.171954 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:27.172133 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:27.173527 (+ 1394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:27.173535 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:27.173578 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:27.173946 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.173959 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.175878 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.175884 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.177793 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.177808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.179091 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.179098 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.179334 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.179391 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 15:07:27.179501 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 15:07:27.179700 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12185,"mutex_wait_us":1,"spinlock_wait_cycles":768,"thread_start_us":13915,"threads_started":1}
I20260808 15:07:27.180343 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.159633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55444 (local address 127.2.180.195:41231)
0808 15:07:27.172588 (+ 12955us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.172598 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.172639 (+ 41us) server_negotiation.cc:407] Connection header received
0808 15:07:27.172750 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.172758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.173064 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.174192 (+ 1128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:27.174474 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.175739 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.177981 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.178891 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.179011 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.179353 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.179612 (+ 259us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.179791 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12760,"thread_start_us":13599,"threads_started":1}
I20260808 15:07:27.181931 520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.159566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49124)
0808 15:07:27.173332 (+ 13766us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.173352 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:27.173581 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:27.174489 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:27.174493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:27.174517 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:27.174957 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.174967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.176867 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.176871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.179132 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.179146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.181142 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.181158 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.181319 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.181351 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 15:07:27.181435 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 15:07:27.181554 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13643,"mutex_wait_us":7,"spinlock_wait_cycles":1920,"thread_start_us":11940,"threads_started":1}
I20260808 15:07:27.182717 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.161373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:42445 (local address 127.0.0.1:49578)
0808 15:07:27.173613 (+ 12240us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.173628 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:27.173755 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:27.174469 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:27.174473 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:27.174499 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:27.175005 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.175019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.176863 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.176869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.179371 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.179384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.181904 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.181912 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.182126 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.182157 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 15:07:27.182255 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 15:07:27.182378 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12132,"mutex_wait_us":370,"spinlock_wait_cycles":7040,"thread_start_us":11680,"threads_started":1}
I20260808 15:07:27.182783 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.162082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49578 (local address 127.2.180.194:42445)
0808 15:07:27.172881 (+ 10799us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.172891 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.173877 (+ 986us) server_negotiation.cc:407] Connection header received
0808 15:07:27.173979 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.173985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.174168 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.174341 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:27.175203 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.176713 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.179559 (+ 2846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.181421 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.181544 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.181758 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.182322 (+ 564us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.182480 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10631,"thread_start_us":10517,"threads_started":1}
I20260808 15:07:27.183276 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.159881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49124 (local address 127.2.180.193:43625)
0808 15:07:27.171174 (+ 11293us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.171184 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.173463 (+ 2279us) server_negotiation.cc:407] Connection header received
0808 15:07:27.173792 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.173801 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.174015 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.174181 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:27.175105 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.176703 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.179364 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.180972 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.181100 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.181333 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.181487 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.181646 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11096,"thread_start_us":11290,"threads_started":1}
I20260808 15:07:27.184870 390 tablet_service.cc:1467] Processing CreateTablet for tablet c733ca00a8d345718ae22127f280e57a (DEFAULT_TABLE table=test-table [id=690e288f00d1446db4a60c2401399f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:27.186239 390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c733ca00a8d345718ae22127f280e57a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:27.186262 32670 tablet_service.cc:1467] Processing CreateTablet for tablet c733ca00a8d345718ae22127f280e57a (DEFAULT_TABLE table=test-table [id=690e288f00d1446db4a60c2401399f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:27.186628 32476 tablet_service.cc:1467] Processing CreateTablet for tablet c733ca00a8d345718ae22127f280e57a (DEFAULT_TABLE table=test-table [id=690e288f00d1446db4a60c2401399f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 15:07:27.187637 32670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c733ca00a8d345718ae22127f280e57a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:27.187969 32476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c733ca00a8d345718ae22127f280e57a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:27.197381 522 tablet_bootstrap.cc:492] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: Bootstrap starting.
I20260808 15:07:27.200439 522 tablet_bootstrap.cc:654] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:27.201510 522 log.cc:824] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:27.201697 523 tablet_bootstrap.cc:492] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: Bootstrap starting.
I20260808 15:07:27.205286 523 tablet_bootstrap.cc:654] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:27.206077 526 tablet_bootstrap.cc:492] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: Bootstrap starting.
I20260808 15:07:27.206502 523 log.cc:824] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:27.209069 522 tablet_bootstrap.cc:492] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: No bootstrap required, opened a new log
I20260808 15:07:27.209367 522 ts_tablet_manager.cc:1397] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260808 15:07:27.209570 526 tablet_bootstrap.cc:654] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:27.210232 523 tablet_bootstrap.cc:492] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: No bootstrap required, opened a new log
I20260808 15:07:27.210484 523 ts_tablet_manager.cc:1397] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 15:07:27.210474 526 log.cc:824] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:27.213748 526 tablet_bootstrap.cc:492] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: No bootstrap required, opened a new log
I20260808 15:07:27.213970 526 ts_tablet_manager.cc:1397] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260808 15:07:27.219597 522 raft_consensus.cc:348] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.220067 522 raft_consensus.cc:374] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:27.220192 522 raft_consensus.cc:729] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17a11551a5614288afcce7d86c01db1b, State: Initialized, Role: FOLLOWER
I20260808 15:07:27.220476 523 raft_consensus.cc:348] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.220886 523 raft_consensus.cc:374] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:27.220994 523 raft_consensus.cc:729] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23d1dccb22ba484fb44d27e0a546e277, State: Initialized, Role: FOLLOWER
I20260808 15:07:27.220944 522 consensus_queue.cc:260] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.221561 523 consensus_queue.cc:260] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.224514 526 raft_consensus.cc:348] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.224938 526 raft_consensus.cc:374] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:27.225064 526 raft_consensus.cc:729] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8adc7f5c772948c8887274592cfe9617, State: Initialized, Role: FOLLOWER
I20260808 15:07:27.225795 526 consensus_queue.cc:260] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.226815 535 raft_consensus.cc:482] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 15:07:27.226905 497 heartbeater.cc:503] Master 127.2.180.254:44035 was elected leader, sending a full tablet report...
I20260808 15:07:27.227563 535 raft_consensus.cc:504] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.234153 526 ts_tablet_manager.cc:1428] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: Time spent starting tablet: real 0.020s user 0.008s sys 0.005s
I20260808 15:07:27.241032 535 leader_election.cc:288] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8adc7f5c772948c8887274592cfe9617 (127.2.180.193:43625), 23d1dccb22ba484fb44d27e0a546e277 (127.2.180.194:42445)
I20260808 15:07:27.241405 32571 heartbeater.cc:503] Master 127.2.180.254:44035 was elected leader, sending a full tablet report...
I20260808 15:07:27.242219 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.233061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:45571 (local address 127.2.180.193:43625)
0808 15:07:27.233460 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.233467 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.233486 (+ 19us) server_negotiation.cc:407] Connection header received
0808 15:07:27.233548 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.233553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.233641 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.233747 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:27.234810 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.235878 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.238735 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.240027 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.240182 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.240256 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.241868 (+ 1612us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.241988 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268}
I20260808 15:07:27.241758 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.232041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.2.180.195:45571)
0808 15:07:27.232231 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.232253 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:27.232379 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:27.233919 (+ 1540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:27.233924 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:27.233951 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:27.234186 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.234200 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.236071 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.236077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.238236 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.238253 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.240993 (+ 2740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.241000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.241223 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.241256 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 15:07:27.241347 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 15:07:27.241486 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 15:07:27.245925 32507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c733ca00a8d345718ae22127f280e57a" candidate_uuid: "17a11551a5614288afcce7d86c01db1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8adc7f5c772948c8887274592cfe9617" is_pre_election: true
I20260808 15:07:27.246457 32507 raft_consensus.cc:2454] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 17a11551a5614288afcce7d86c01db1b in term 0.
I20260808 15:07:27.247332 32760 heartbeater.cc:503] Master 127.2.180.254:44035 was elected leader, sending a full tablet report...
I20260808 15:07:27.247901 523 ts_tablet_manager.cc:1428] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: Time spent starting tablet: real 0.037s user 0.021s sys 0.016s
I20260808 15:07:27.247833 322 leader_election.cc:302] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a11551a5614288afcce7d86c01db1b, 8adc7f5c772948c8887274592cfe9617; no voters:
I20260808 15:07:27.248154 522 ts_tablet_manager.cc:1428] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: Time spent starting tablet: real 0.038s user 0.019s sys 0.016s
I20260808 15:07:27.249698 531 raft_consensus.cc:2781] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 15:07:27.249869 531 raft_consensus.cc:482] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 15:07:27.250011 531 raft_consensus.cc:3037] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:27.253427 531 raft_consensus.cc:504] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.253789 499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.234718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.194:42445 (local address 127.2.180.195:49395)
0808 15:07:27.245180 (+ 10462us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.245197 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:07:27.245293 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:07:27.248318 (+ 3025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:07:27.248323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:07:27.248348 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 15:07:27.248552 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.248563 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.249827 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.249832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.251623 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:07:27.251638 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.253181 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.253186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:07:27.253339 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.253362 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 15:07:27.253439 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 15:07:27.253522 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10354,"spinlock_wait_cycles":2304,"thread_start_us":14001,"threads_started":1}
I20260808 15:07:27.253841 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.235171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.2.180.195:49395 (local address 127.2.180.194:42445)
0808 15:07:27.235351 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.235357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.246902 (+ 11545us) server_negotiation.cc:407] Connection header received
0808 15:07:27.246984 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.246989 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.247077 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.247173 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 15:07:27.248745 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.249661 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.251881 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.253025 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.253176 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.253235 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.253490 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.253616 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 15:07:27.254142 531 leader_election.cc:288] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [CANDIDATE]: Term 1 election: Requested vote from peers 8adc7f5c772948c8887274592cfe9617 (127.2.180.193:43625), 23d1dccb22ba484fb44d27e0a546e277 (127.2.180.194:42445)
I20260808 15:07:27.254477 32507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c733ca00a8d345718ae22127f280e57a" candidate_uuid: "17a11551a5614288afcce7d86c01db1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8adc7f5c772948c8887274592cfe9617"
I20260808 15:07:27.254694 32507 raft_consensus.cc:3037] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:27.254834 32697 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c733ca00a8d345718ae22127f280e57a" candidate_uuid: "17a11551a5614288afcce7d86c01db1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d1dccb22ba484fb44d27e0a546e277"
I20260808 15:07:27.254791 32700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c733ca00a8d345718ae22127f280e57a" candidate_uuid: "17a11551a5614288afcce7d86c01db1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23d1dccb22ba484fb44d27e0a546e277" is_pre_election: true
I20260808 15:07:27.255182 32697 raft_consensus.cc:3037] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:27.257340 32507 raft_consensus.cc:2454] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a11551a5614288afcce7d86c01db1b in term 1.
I20260808 15:07:27.257706 322 leader_election.cc:302] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 17a11551a5614288afcce7d86c01db1b, 8adc7f5c772948c8887274592cfe9617; no voters:
I20260808 15:07:27.257779 32697 raft_consensus.cc:2454] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 17a11551a5614288afcce7d86c01db1b in term 1.
I20260808 15:07:27.257961 531 raft_consensus.cc:2781] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:27.258149 531 raft_consensus.cc:686] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 1 LEADER]: Becoming Leader. State: Replica: 17a11551a5614288afcce7d86c01db1b, State: Running, Role: LEADER
I20260808 15:07:27.258558 531 consensus_queue.cc:237] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } }
I20260808 15:07:27.262068 32288 catalog_manager.cc:5495] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b reported cstate change: term changed from 0 to 1, leader changed from <none> to 17a11551a5614288afcce7d86c01db1b (127.2.180.195). New cstate: current_term: 1 leader_uuid: "17a11551a5614288afcce7d86c01db1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17a11551a5614288afcce7d86c01db1b" member_type: VOTER last_known_addr { host: "127.2.180.195" port: 41231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 } health_report { overall_health: UNKNOWN } } }
I20260808 15:07:27.286074 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.281092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55446 (local address 127.2.180.195:41231)
0808 15:07:27.281286 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.281292 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.281383 (+ 91us) server_negotiation.cc:407] Connection header received
0808 15:07:27.281577 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.281582 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.281646 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.281764 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:07:27.282107 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.282803 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.284206 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.284599 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.284703 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.285144 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:07:27.285558 (+ 414us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:07:27.285680 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.285790 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.285863 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:07:27.292811 32697 raft_consensus.cc:1261] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [term 1 FOLLOWER]: Refusing update from remote peer 17a11551a5614288afcce7d86c01db1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:27.292899 32507 raft_consensus.cc:1261] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 1 FOLLOWER]: Refusing update from remote peer 17a11551a5614288afcce7d86c01db1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:07:27.293525 535 consensus_queue.cc:1035] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "23d1dccb22ba484fb44d27e0a546e277" member_type: VOTER last_known_addr { host: "127.2.180.194" port: 42445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:27.293797 531 consensus_queue.cc:1035] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8adc7f5c772948c8887274592cfe9617" member_type: VOTER last_known_addr { host: "127.2.180.193" port: 43625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:07:27.303642 544 mvcc.cc:205] Tried to move back new op lower bound from 7316281947303587840 to 7316281947174232064. Current Snapshot: MvccSnapshot[applied={T|T < 7316281947303587840}]
I20260808 15:07:27.304762 545 mvcc.cc:205] Tried to move back new op lower bound from 7316281947303587840 to 7316281947174232064. Current Snapshot: MvccSnapshot[applied={T|T < 7316281947303587840}]
I20260808 15:07:27.316040 32289 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:27.316287 32289 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 15:07:27.319984 32289 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6a0a83afd3be47d2adc310893a9730f6: Sending DeleteTablet for 3 replicas of tablet c733ca00a8d345718ae22127f280e57a
I20260808 15:07:27.320914 390 tablet_service.cc:1514] Processing DeleteTablet for tablet c733ca00a8d345718ae22127f280e57a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:27 UTC) from {username='slave'} at 127.0.0.1:55444
I20260808 15:07:27.321046 32670 tablet_service.cc:1514] Processing DeleteTablet for tablet c733ca00a8d345718ae22127f280e57a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:27 UTC) from {username='slave'} at 127.0.0.1:49578
I20260808 15:07:27.321063 32476 tablet_service.cc:1514] Processing DeleteTablet for tablet c733ca00a8d345718ae22127f280e57a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 15:07:27 UTC) from {username='slave'} at 127.0.0.1:49124
I20260808 15:07:27.322685 563 tablet_replica.cc:330] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: stopping tablet replica
I20260808 15:07:27.322727 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 32385
I20260808 15:07:27.323141 563 raft_consensus.cc:2229] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:27.323648 563 raft_consensus.cc:2258] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:27.326115 563 ts_tablet_manager.cc:1905] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:27.332937 563 ts_tablet_manager.cc:1918] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:27.333272 565 tablet_replica.cc:330] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: stopping tablet replica
I20260808 15:07:27.333709 565 raft_consensus.cc:2229] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 15:07:27.334237 565 raft_consensus.cc:2258] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:27.336395 565 ts_tablet_manager.cc:1905] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:27.336464 562 tablet_replica.cc:330] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: stopping tablet replica
I20260808 15:07:27.336845 562 raft_consensus.cc:2229] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 1 LEADER]: Raft consensus shutting down.
I20260808 15:07:27.337448 562 raft_consensus.cc:2258] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 15:07:27.339602 562 ts_tablet_manager.cc:1905] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 15:07:27.343654 565 ts_tablet_manager.cc:1918] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:27.343895 563 log.cc:1196] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-1/wal/wals/c733ca00a8d345718ae22127f280e57a
I20260808 15:07:27.344493 563 ts_tablet_manager.cc:1939] T c733ca00a8d345718ae22127f280e57a P 23d1dccb22ba484fb44d27e0a546e277: Deleting consensus metadata
I20260808 15:07:27.345121 517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.324550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49130 (local address 127.2.180.193:43625)
0808 15:07:27.324726 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.324731 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.337213 (+ 12482us) server_negotiation.cc:407] Connection header received
0808 15:07:27.337630 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.337636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.337749 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.337853 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:27.338521 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.339634 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.341402 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.342095 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.342195 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.343517 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:27.343527 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:27.343542 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:27.344285 (+ 743us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:27.344338 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:27.344362 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:27.344438 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.344684 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.344796 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 15:07:27.346535 32270 catalog_manager.cc:4841] TS 23d1dccb22ba484fb44d27e0a546e277 (127.2.180.194:42445): tablet c733ca00a8d345718ae22127f280e57a (table test-table [id=690e288f00d1446db4a60c2401399f67]) successfully deleted
I20260808 15:07:27.346949 32566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:27.346953 562 ts_tablet_manager.cc:1918] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 15:07:27.347079 562 log.cc:1196] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-2/wal/wals/c733ca00a8d345718ae22127f280e57a
I20260808 15:07:27.347586 562 ts_tablet_manager.cc:1939] T c733ca00a8d345718ae22127f280e57a P 17a11551a5614288afcce7d86c01db1b: Deleting consensus metadata
I20260808 15:07:27.349507 32272 catalog_manager.cc:4841] TS 17a11551a5614288afcce7d86c01db1b (127.2.180.195:41231): tablet c733ca00a8d345718ae22127f280e57a (table test-table [id=690e288f00d1446db4a60c2401399f67]) successfully deleted
I20260808 15:07:27.350471 565 log.cc:1196] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: Deleting WAL directory at /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786201338129629-2771-0/minicluster-data/ts-0/wal/wals/c733ca00a8d345718ae22127f280e57a
I20260808 15:07:27.434333 565 ts_tablet_manager.cc:1939] T c733ca00a8d345718ae22127f280e57a P 8adc7f5c772948c8887274592cfe9617: Deleting consensus metadata
I20260808 15:07:27.434485 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 32385
W20260808 15:07:27.455391 32268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.2.180.193:43625 (error 108)
I20260808 15:07:27.455839 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 32575
I20260808 15:07:27.457547 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.457017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49132)
0808 15:07:27.457229 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.457364 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260808 15:07:27.457831 32268 catalog_manager.cc:4584] TS 8adc7f5c772948c8887274592cfe9617 (127.2.180.193:43625): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c733ca00a8d345718ae22127f280e57a: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
I20260808 15:07:27.462927 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.456703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49580 (local address 127.2.180.194:42445)
0808 15:07:27.456896 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.456902 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.456924 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:07:27.457109 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.457115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.457222 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.457321 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:27.457972 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.458803 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.460137 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.460522 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.460631 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.461671 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:27.461680 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:27.461695 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:27.462247 (+ 552us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:27.462293 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:27.462312 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:27.462407 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.462536 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.462644 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:07:27.463604 32754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:27.511547 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.510991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49136)
0808 15:07:27.511241 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.511345 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 15:07:27.512748 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.512313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49148)
0808 15:07:27.512477 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.512592 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 15:07:27.549141 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 32575
I20260808 15:07:27.573330 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.572860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49158)
0808 15:07:27.573071 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.573167 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 15:07:27.574350 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.574005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49174)
0808 15:07:27.574147 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.574219 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 15:07:27.581679 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 300
I20260808 15:07:27.588815 515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.582373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.2.180.195:41231)
0808 15:07:27.582540 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:27.582546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:27.582640 (+ 94us) server_negotiation.cc:407] Connection header received
0808 15:07:27.582835 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:27.582839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:27.582938 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:27.583040 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:27.583611 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.584487 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.585744 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:27.586193 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:27.586307 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:27.587383 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:27.587391 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:27.587407 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:27.588110 (+ 703us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:27.588158 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:27.588178 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:27.588271 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:27.588412 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 15:07:27.588508 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:07:27.589506 492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:27.654958 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.654386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49180)
0808 15:07:27.654660 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.654767 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 15:07:27.656087 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.655603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49182)
0808 15:07:27.655830 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.655950 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 15:07:27.705188 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 300
I20260808 15:07:27.729817 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 32245
I20260808 15:07:27.730854 32348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:27.838657 513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:27.838041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.193:43625 (local address 127.0.0.1:49198)
0808 15:07:27.838349 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 15:07:27.838461 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.193:43625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260808 15:07:27.838642 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 32245
2026-08-08T15:07:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2652 ms)
[----------] 6 tests from EncryptionPolicyTest (20969 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:27 dist-test-slave-jlzn krb5kdc[580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:27 dist-test-slave-jlzn krb5kdc[580](info): set up 2 sockets
Aug 08 15:07:27 dist-test-slave-jlzn krb5kdc[580](info): commencing operation
krb5kdc: starting...
W20260808 15:07:29.019743 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:29 dist-test-slave-jlzn krb5kdc[580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:29.260222 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40515
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:29.572926 597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:29.573172 597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:29.573267 597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:29.580094 597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:29.580188 597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:29.580237 597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:29.580296 597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:29.580339 597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:29.589463 597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40515
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:29.590480 597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:29.591639 597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:29.602454 607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:29.602500 612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:29.602527 608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:29.603466 597 server_base.cc:1034] running on GCE node
I20260808 15:07:29.604175 597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:29.605370 597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:29.606557 597 hybrid_clock.cc:648] HybridClock initialized: now 1786201649606521 us; error 31 us; skew 500 ppm
I20260808 15:07:29.606875 597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:29 dist-test-slave-jlzn krb5kdc[580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201649, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:29.612497 597 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:29.613880 597 webserver.cc:460] Webserver started at http://127.0.0.1:38889/ using document root <none> and password file <none>
I20260808 15:07:29.614494 597 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:29.614614 597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:29.614900 597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:29.617693 597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "86962147b5d74fba9d931ce0fdf9f76d"
format_stamp: "Formatted at 2026-08-08 15:07:29 on dist-test-slave-jlzn"
server_key: "70c3d29bc1b540aff6139c53e47f3a46"
server_key_iv: "96e9f6761027c670ace782adb03215bb"
server_key_version: "encryptionkey@0"
I20260808 15:07:29.618417 597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "86962147b5d74fba9d931ce0fdf9f76d"
format_stamp: "Formatted at 2026-08-08 15:07:29 on dist-test-slave-jlzn"
server_key: "70c3d29bc1b540aff6139c53e47f3a46"
server_key_iv: "96e9f6761027c670ace782adb03215bb"
server_key_version: "encryptionkey@0"
I20260808 15:07:29.623103 597 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:07:29.626335 620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:29.627460 597 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:07:29.627614 597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "86962147b5d74fba9d931ce0fdf9f76d"
format_stamp: "Formatted at 2026-08-08 15:07:29 on dist-test-slave-jlzn"
server_key: "70c3d29bc1b540aff6139c53e47f3a46"
server_key_iv: "96e9f6761027c670ace782adb03215bb"
server_key_version: "encryptionkey@0"
I20260808 15:07:29.627785 597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:29.652647 597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:29.656198 597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:29.656432 597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:29.673080 597 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40515
I20260808 15:07:29.673110 695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40515 every 8 connection(s)
I20260808 15:07:29.674309 597 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:29.677757 696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:29.679963 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 597
I20260808 15:07:29.680166 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:29.680991 2771 external_mini_cluster.cc:1428] Setting key 5ae9f8b1eb9f6a85dc39b679ce55106c
I20260808 15:07:29.690021 696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d: Bootstrap starting.
I20260808 15:07:29.693632 696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:29.694890 696 log.cc:824] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:07:29 dist-test-slave-jlzn krb5kdc[580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201649, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 15:07:29.697839 696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d: No bootstrap required, opened a new log
I20260808 15:07:29.703402 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:29.682952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55985 (local address 127.0.0.1:40515)
0808 15:07:29.683741 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:29.683752 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:29.683791 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:07:29.685427 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:29.685448 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:29.685801 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:29.686096 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:29.687892 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:29.689165 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:29.690966 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:29.691620 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:29.691749 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:29.696644 (+ 4895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:29.696652 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:29.696665 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:29.696710 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:29.699548 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:29.700328 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:29.700334 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:29.700339 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:29.700504 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:29.700956 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:29.700960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:29.700962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:29.701426 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:29.701751 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:29.702208 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 15:07:29.702477 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":365,"threads_started":1}
I20260808 15:07:29.705991 696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86962147b5d74fba9d931ce0fdf9f76d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } }
I20260808 15:07:29.706326 696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:29.706424 696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86962147b5d74fba9d931ce0fdf9f76d, State: Initialized, Role: FOLLOWER
I20260808 15:07:29.706929 696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86962147b5d74fba9d931ce0fdf9f76d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } }
I20260808 15:07:29.707208 696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:29.707221 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:29.707332 696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:29.707484 696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:29.709487 696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86962147b5d74fba9d931ce0fdf9f76d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } }
I20260808 15:07:29.709952 696 leader_election.cc:302] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86962147b5d74fba9d931ce0fdf9f76d; no voters:
I20260808 15:07:29.710719 696 leader_election.cc:288] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:29.710811 708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:29.712209 708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [term 1 LEADER]: Becoming Leader. State: Replica: 86962147b5d74fba9d931ce0fdf9f76d, State: Running, Role: LEADER
I20260808 15:07:29.712852 696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:29.712783 708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86962147b5d74fba9d931ce0fdf9f76d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } }
I20260808 15:07:29.715194 709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86962147b5d74fba9d931ce0fdf9f76d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86962147b5d74fba9d931ce0fdf9f76d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } } }
I20260808 15:07:29.715569 709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:29.719928 710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86962147b5d74fba9d931ce0fdf9f76d. Latest consensus state: current_term: 1 leader_uuid: "86962147b5d74fba9d931ce0fdf9f76d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86962147b5d74fba9d931ce0fdf9f76d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40515 } } }
I20260808 15:07:29.720248 710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:29.721802 717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:29.726676 717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:29.730245 701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:29.712059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40061 (local address 127.0.0.1:40515)
0808 15:07:29.712198 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:29.712204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:29.712666 (+ 462us) server_negotiation.cc:407] Connection header received
0808 15:07:29.712867 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:29.712872 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:29.712951 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:29.713050 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:29.715063 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:29.716420 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:29.718311 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:29.719036 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:29.719127 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:29.722407 (+ 3280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:29.722417 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:29.722421 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:29.722456 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:29.726074 (+ 3618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:29.727100 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:29.727104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:29.727107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:29.727260 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:29.727890 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:29.727894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:29.727897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:29.728328 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:29.729529 (+ 1201us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:29.729725 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 15:07:29.729903 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 15:07:29.735486 717 catalog_manager.cc:1347] Generated new cluster ID: d543a3f23efd4915a70718b403883445
I20260808 15:07:29.735581 717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:29.764308 717 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:29.765489 717 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:29.772632 717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86962147b5d74fba9d931ce0fdf9f76d: Generated new TSK 0
I20260808 15:07:29.773305 717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:29.788029 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 597
I20260808 15:07:29.789160 686 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:29.880751 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 597
W20260808 15:07:29.903270 578 connection.cc:535] client connection to 127.0.0.1:40515 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40515): Connection reset by peer (error 104)
2026-08-08T15:07:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2039 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:29 dist-test-slave-jlzn krb5kdc[738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:29 dist-test-slave-jlzn krb5kdc[738](info): set up 2 sockets
Aug 08 15:07:29 dist-test-slave-jlzn krb5kdc[738](info): commencing operation
krb5kdc: starting...
W20260808 15:07:31.068819 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:31 dist-test-slave-jlzn krb5kdc[738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:31.312073 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:31.630366 754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:31.630617 754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:31.630715 754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:31.637673 754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:31.637773 754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:31.637825 754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:31.637883 754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:31.637925 754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:31.647042 754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:31.648100 754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:31.649281 754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:31.660137 762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:31.660104 763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:31.660673 765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:31.661118 754 server_base.cc:1034] running on GCE node
I20260808 15:07:31.661792 754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:31.662995 754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:31.664192 754 hybrid_clock.cc:648] HybridClock initialized: now 1786201651664150 us; error 28 us; skew 500 ppm
I20260808 15:07:31.664507 754 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:31 dist-test-slave-jlzn krb5kdc[738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201651, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:31.667480 754 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:31.668869 754 webserver.cc:460] Webserver started at http://127.0.0.1:39543/ using document root <none> and password file <none>
I20260808 15:07:31.669625 754 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:31.669767 754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:31.670109 754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:31.672964 754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "93e0d326c89741209a94f9bc4c22b761"
format_stamp: "Formatted at 2026-08-08 15:07:31 on dist-test-slave-jlzn"
server_key: "e5481273aab654e0195073897bf98aac"
server_key_iv: "12fafc1f38d3a695190d7070febd1a6e"
server_key_version: "encryptionkey@0"
I20260808 15:07:31.673684 754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "93e0d326c89741209a94f9bc4c22b761"
format_stamp: "Formatted at 2026-08-08 15:07:31 on dist-test-slave-jlzn"
server_key: "e5481273aab654e0195073897bf98aac"
server_key_iv: "12fafc1f38d3a695190d7070febd1a6e"
server_key_version: "encryptionkey@0"
I20260808 15:07:31.678419 754 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:07:31.681646 777 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:31.682799 754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:07:31.682948 754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "93e0d326c89741209a94f9bc4c22b761"
format_stamp: "Formatted at 2026-08-08 15:07:31 on dist-test-slave-jlzn"
server_key: "e5481273aab654e0195073897bf98aac"
server_key_iv: "12fafc1f38d3a695190d7070febd1a6e"
server_key_version: "encryptionkey@0"
I20260808 15:07:31.683126 754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:31.724083 754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:31.728030 754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:31.728282 754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:31.745079 754 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33191
I20260808 15:07:31.745098 850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33191 every 8 connection(s)
I20260808 15:07:31.746304 754 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:31.749840 852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:31.751739 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 754
I20260808 15:07:31.751943 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:31.752480 2771 external_mini_cluster.cc:1428] Setting key cf623859809c7eca337a59a351d3a086
I20260808 15:07:31.763446 852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761: Bootstrap starting.
I20260808 15:07:31.765359 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:31.754143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44377 (local address 127.0.0.1:33191)
0808 15:07:31.754910 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:31.754921 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:31.754963 (+ 42us) server_negotiation.cc:407] Connection header received
0808 15:07:31.756983 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:31.757007 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:31.757366 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:31.757653 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:31.758199 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:31.759421 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:31.761272 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:31.761907 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:31.762027 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:31.763218 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:31.763225 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:31.763237 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:31.763772 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:31.763882 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:31.763909 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:31.764021 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:31.764291 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 15:07:31.764445 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":345,"threads_started":1}
I20260808 15:07:31.766525 852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:31.767581 852 log.cc:824] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:31.768906 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:31.770205 852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761: No bootstrap required, opened a new log
I20260808 15:07:31.777786 852 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e0d326c89741209a94f9bc4c22b761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33191 } }
I20260808 15:07:31.778127 852 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:31.778229 852 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93e0d326c89741209a94f9bc4c22b761, State: Initialized, Role: FOLLOWER
I20260808 15:07:31.778692 855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:31.772954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50387 (local address 127.0.0.1:33191)
0808 15:07:31.773105 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:31.773111 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:31.773458 (+ 347us) server_negotiation.cc:407] Connection header received
0808 15:07:31.773622 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:31.773626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:31.773695 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:31.773779 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:31.774276 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:31.775039 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:31.776365 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:31.776821 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:31.776945 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:31.777931 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:31.777936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:31.777939 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:31.777954 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:31.778008 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:31.778027 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:31.778132 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:31.778353 (+ 221us) server_negotiation.cc:299] Negotiation successful
0808 15:07:31.778401 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 15:07:31.778851 852 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e0d326c89741209a94f9bc4c22b761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33191 } }
I20260808 15:07:31.779162 852 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:31.779315 852 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:31.779479 852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:31.781613 852 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e0d326c89741209a94f9bc4c22b761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33191 } }
I20260808 15:07:31.782058 852 leader_election.cc:302] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93e0d326c89741209a94f9bc4c22b761; no voters:
I20260808 15:07:31.782550 852 leader_election.cc:288] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:31.782662 864 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:31.783318 864 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [term 1 LEADER]: Becoming Leader. State: Replica: 93e0d326c89741209a94f9bc4c22b761, State: Running, Role: LEADER
I20260808 15:07:31.783841 864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e0d326c89741209a94f9bc4c22b761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33191 } }
I20260808 15:07:31.784437 852 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:31.786279 866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93e0d326c89741209a94f9bc4c22b761. Latest consensus state: current_term: 1 leader_uuid: "93e0d326c89741209a94f9bc4c22b761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e0d326c89741209a94f9bc4c22b761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33191 } } }
I20260808 15:07:31.786672 866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:31.787199 865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93e0d326c89741209a94f9bc4c22b761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93e0d326c89741209a94f9bc4c22b761" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33191 } } }
I20260808 15:07:31.787518 865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:31.788264 869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:31.793455 869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:31.801903 869 catalog_manager.cc:1347] Generated new cluster ID: 2dba43f224104b739153a6190eaab928
I20260808 15:07:31.802004 869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:31.814255 869 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:31.815362 869 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:31.828696 869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93e0d326c89741209a94f9bc4c22b761: Generated new TSK 0
I20260808 15:07:31.829695 869 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:31.843465 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 754
W20260808 15:07:31.844877 842 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:44377
W20260808 15:07:31.845553 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 754: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:31.845733 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 754
W20260808 15:07:31.869875 736 connection.cc:535] client connection to 127.0.0.1:33191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33191): Connection reset by peer (error 104)
2026-08-08T15:07:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1965 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:31 dist-test-slave-jlzn krb5kdc[893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:31 dist-test-slave-jlzn krb5kdc[893](info): set up 2 sockets
Aug 08 15:07:31 dist-test-slave-jlzn krb5kdc[893](info): commencing operation
krb5kdc: starting...
W20260808 15:07:33.041013 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.004s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:33 dist-test-slave-jlzn krb5kdc[893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:33.274731 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35259
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:33.591639 909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:33.591917 909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:33.592043 909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:33.599195 909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:33.599290 909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:33.599363 909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:33.599443 909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:33.599503 909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:33.608573 909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35259
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46813
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:33.609671 909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:33.610859 909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:33.621690 919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:33.621730 923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:33.622443 925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:33.622992 909 server_base.cc:1034] running on GCE node
I20260808 15:07:33.623926 909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:33.625597 909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:33.626842 909 hybrid_clock.cc:648] HybridClock initialized: now 1786201653626789 us; error 38 us; skew 500 ppm
I20260808 15:07:33.627239 909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:33 dist-test-slave-jlzn krb5kdc[893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201653, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:33.632555 909 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:33.633859 909 webserver.cc:460] Webserver started at http://127.0.0.1:46247/ using document root <none> and password file <none>
I20260808 15:07:33.634569 909 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:33.634696 909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:33.634943 909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:33.637790 909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "ce2882415cf044178b7c5372040daad4"
format_stamp: "Formatted at 2026-08-08 15:07:33 on dist-test-slave-jlzn"
server_key: "2cbcd452d0d9894cb431af1c5ceed980"
server_key_iv: "b06e0c73531b81c13201f6935c47f8a8"
server_key_version: "encryptionkey@0"
I20260808 15:07:33.638473 909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "ce2882415cf044178b7c5372040daad4"
format_stamp: "Formatted at 2026-08-08 15:07:33 on dist-test-slave-jlzn"
server_key: "2cbcd452d0d9894cb431af1c5ceed980"
server_key_iv: "b06e0c73531b81c13201f6935c47f8a8"
server_key_version: "encryptionkey@0"
I20260808 15:07:33.643250 909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 15:07:33.646368 933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:33.647505 909 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:07:33.647645 909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "ce2882415cf044178b7c5372040daad4"
format_stamp: "Formatted at 2026-08-08 15:07:33 on dist-test-slave-jlzn"
server_key: "2cbcd452d0d9894cb431af1c5ceed980"
server_key_iv: "b06e0c73531b81c13201f6935c47f8a8"
server_key_version: "encryptionkey@0"
I20260808 15:07:33.647799 909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:33.652945 909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:33.656469 909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:33.656697 909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:33.672864 909 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46813
I20260808 15:07:33.672905 1010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46813 every 8 connection(s)
I20260808 15:07:33.674098 909 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:33.677400 1015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:33.683494 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 909
I20260808 15:07:33.683682 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:33.684216 2771 external_mini_cluster.cc:1428] Setting key 0696fe78faf3a3669e1b853676c4f3aa
I20260808 15:07:33.687463 1015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4: Bootstrap starting.
I20260808 15:07:33.690649 1015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:33.691435 1019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:33.685871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45319 (local address 127.0.0.1:46813)
0808 15:07:33.686606 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:33.686617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:33.686658 (+ 41us) server_negotiation.cc:407] Connection header received
0808 15:07:33.688367 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:33.688389 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:33.688704 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:33.689021 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:33.690025 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:33.690031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:33.690043 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:33.690056 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:33.690113 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:33.690132 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:33.690273 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:33.690413 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 15:07:33.690573 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":327,"threads_started":1}
I20260808 15:07:33.691695 1015 log.cc:824] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:33.694391 1015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4: No bootstrap required, opened a new log
I20260808 15:07:33.694787 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:33.701010 1019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:33.698804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42471 (local address 127.0.0.1:46813)
0808 15:07:33.698944 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:33.698950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:33.699308 (+ 358us) server_negotiation.cc:407] Connection header received
0808 15:07:33.699457 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:33.699461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:33.699502 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:33.699583 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:33.700420 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:33.700426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:33.700430 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:33.700446 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:33.700486 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:33.700501 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:33.700602 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:33.700716 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 15:07:33.700770 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 15:07:33.701859 1015 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2882415cf044178b7c5372040daad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } }
I20260808 15:07:33.702206 1015 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:33.702309 1015 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce2882415cf044178b7c5372040daad4, State: Initialized, Role: FOLLOWER
I20260808 15:07:33.702781 1015 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2882415cf044178b7c5372040daad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } }
I20260808 15:07:33.702986 1015 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:33.703083 1015 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:33.703204 1015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:33.705164 1015 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2882415cf044178b7c5372040daad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } }
I20260808 15:07:33.705566 1015 leader_election.cc:302] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce2882415cf044178b7c5372040daad4; no voters:
I20260808 15:07:33.706059 1015 leader_election.cc:288] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:33.706153 1031 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:33.706789 1031 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [term 1 LEADER]: Becoming Leader. State: Replica: ce2882415cf044178b7c5372040daad4, State: Running, Role: LEADER
I20260808 15:07:33.707274 1031 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2882415cf044178b7c5372040daad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } }
I20260808 15:07:33.707911 1015 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:33.710212 1032 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce2882415cf044178b7c5372040daad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2882415cf044178b7c5372040daad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } } }
I20260808 15:07:33.710650 1032 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:33.711098 1033 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce2882415cf044178b7c5372040daad4. Latest consensus state: current_term: 1 leader_uuid: "ce2882415cf044178b7c5372040daad4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce2882415cf044178b7c5372040daad4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46813 } } }
I20260808 15:07:33.711386 1033 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:33.713328 1043 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:33.717192 1043 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:33.724939 1043 catalog_manager.cc:1347] Generated new cluster ID: 899ca47ed4204ed585d5c0e7f303ccdf
I20260808 15:07:33.725047 1043 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:33.744753 1043 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:33.745112 1043 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:33.759907 1043 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce2882415cf044178b7c5372040daad4: Generated new TSK 0
I20260808 15:07:33.760601 1043 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:33.772378 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 909
W20260808 15:07:33.773334 1006 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45319
W20260808 15:07:33.773942 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 909: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:33.774061 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 909
2026-08-08T15:07:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1929 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:33 dist-test-slave-jlzn krb5kdc[1064](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:33 dist-test-slave-jlzn krb5kdc[1064](info): set up 2 sockets
Aug 08 15:07:33 dist-test-slave-jlzn krb5kdc[1064](info): commencing operation
krb5kdc: starting...
W20260808 15:07:34.963595 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.145s user 0.004s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:35 dist-test-slave-jlzn krb5kdc[1064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:35.201047 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38913
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:35.515100 1082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:35.515372 1082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:35.515503 1082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:35.522550 1082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:35.522661 1082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:35.522740 1082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:35.522823 1082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:35.522883 1082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:35.532379 1082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38913
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:35.533556 1082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:35.534798 1082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:35.546051 1090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:35.546098 1093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:35.546123 1091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:35.546875 1082 server_base.cc:1034] running on GCE node
I20260808 15:07:35.547573 1082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:35.548875 1082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:35.550071 1082 hybrid_clock.cc:648] HybridClock initialized: now 1786201655550033 us; error 32 us; skew 500 ppm
I20260808 15:07:35.550384 1082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:35 dist-test-slave-jlzn krb5kdc[1064](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201655, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:35.556066 1082 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:35.557399 1082 webserver.cc:460] Webserver started at http://127.0.0.1:41569/ using document root <none> and password file <none>
I20260808 15:07:35.558010 1082 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:35.558113 1082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:35.558350 1082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:35.561157 1082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "76d1b92f740f416780272be7f56572bd"
format_stamp: "Formatted at 2026-08-08 15:07:35 on dist-test-slave-jlzn"
server_key: "9b522ea399311a738c95dafeb8f95c87"
server_key_iv: "5fcb9603768855aa6f0f2720b23bf861"
server_key_version: "encryptionkey@0"
I20260808 15:07:35.561852 1082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "76d1b92f740f416780272be7f56572bd"
format_stamp: "Formatted at 2026-08-08 15:07:35 on dist-test-slave-jlzn"
server_key: "9b522ea399311a738c95dafeb8f95c87"
server_key_iv: "5fcb9603768855aa6f0f2720b23bf861"
server_key_version: "encryptionkey@0"
I20260808 15:07:35.566502 1082 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:07:35.569731 1104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:35.570884 1082 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:07:35.571038 1082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "76d1b92f740f416780272be7f56572bd"
format_stamp: "Formatted at 2026-08-08 15:07:35 on dist-test-slave-jlzn"
server_key: "9b522ea399311a738c95dafeb8f95c87"
server_key_iv: "5fcb9603768855aa6f0f2720b23bf861"
server_key_version: "encryptionkey@0"
I20260808 15:07:35.571208 1082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:35.582293 1082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:35.586241 1082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:35.586483 1082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:35.603896 1082 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32975
I20260808 15:07:35.603946 1174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32975 every 8 connection(s)
I20260808 15:07:35.605178 1082 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:35.608686 1175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:35.610031 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 1082
I20260808 15:07:35.610214 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:35.610767 2771 external_mini_cluster.cc:1428] Setting key b1780489b31b3059a6bff0d492d376ad
I20260808 15:07:35.622175 1175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd: Bootstrap starting.
Aug 08 15:07:35 dist-test-slave-jlzn krb5kdc[1064](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 15:07:35.625666 1175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:35.626688 1175 log.cc:824] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:35.629310 1175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd: No bootstrap required, opened a new log
I20260808 15:07:35.632616 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:35.612448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59627 (local address 127.0.0.1:32975)
0808 15:07:35.613245 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:35.613256 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:35.613300 (+ 44us) server_negotiation.cc:407] Connection header received
0808 15:07:35.614870 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:35.614895 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:35.615254 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:35.615577 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:35.617302 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:35.618767 (+ 1465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:35.620531 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:35.621160 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:35.621285 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:35.625873 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:35.625881 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:35.625894 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:35.625937 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:35.628815 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:35.629483 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:35.629489 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:35.629495 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:35.629670 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:35.630042 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:35.630046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:35.630048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:35.630509 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:35.630845 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:35.631248 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 15:07:35.631640 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":347,"threads_started":1}
I20260808 15:07:35.635941 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:35.636878 1175 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d1b92f740f416780272be7f56572bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32975 } }
I20260808 15:07:35.637192 1175 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:35.637282 1175 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76d1b92f740f416780272be7f56572bd, State: Initialized, Role: FOLLOWER
I20260808 15:07:35.637766 1175 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d1b92f740f416780272be7f56572bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32975 } }
I20260808 15:07:35.637976 1175 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:35.638098 1175 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:35.638252 1175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:35.640772 1175 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d1b92f740f416780272be7f56572bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32975 } }
I20260808 15:07:35.641345 1175 leader_election.cc:302] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76d1b92f740f416780272be7f56572bd; no voters:
I20260808 15:07:35.641952 1175 leader_election.cc:288] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:35.642036 1193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:35.643249 1193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [term 1 LEADER]: Becoming Leader. State: Replica: 76d1b92f740f416780272be7f56572bd, State: Running, Role: LEADER
I20260808 15:07:35.643901 1193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d1b92f740f416780272be7f56572bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32975 } }
I20260808 15:07:35.644210 1175 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:35.646611 1195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76d1b92f740f416780272be7f56572bd. Latest consensus state: current_term: 1 leader_uuid: "76d1b92f740f416780272be7f56572bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d1b92f740f416780272be7f56572bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32975 } } }
I20260808 15:07:35.646585 1194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76d1b92f740f416780272be7f56572bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76d1b92f740f416780272be7f56572bd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32975 } } }
I20260808 15:07:35.647030 1194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:35.647030 1195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:35.647861 1201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:35.657733 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:35.640092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45105 (local address 127.0.0.1:32975)
0808 15:07:35.640284 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:35.640290 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:35.640783 (+ 493us) server_negotiation.cc:407] Connection header received
0808 15:07:35.640952 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:35.640956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:35.641021 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:35.641091 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:35.642970 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:35.644151 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:35.645932 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:35.646680 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:35.646821 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:35.649983 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:35.649991 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:35.649996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:35.650046 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:35.653728 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:35.654609 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:35.654615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:35.654619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:35.654790 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:35.655236 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:35.655242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:35.655245 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:35.655704 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:35.655993 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:35.657137 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0808 15:07:35.657359 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:07:35.664850 1201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:35.673530 1201 catalog_manager.cc:1347] Generated new cluster ID: a8e3f24d5d4041b3924d6ebe3e82fc9c
I20260808 15:07:35.673640 1201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:35.684453 1201 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:35.686103 1201 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:35.700377 1201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76d1b92f740f416780272be7f56572bd: Generated new TSK 0
I20260808 15:07:35.701104 1201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:35.706533 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1082
I20260808 15:07:35.707556 1170 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:35.794500 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 1082
2026-08-08T15:07:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2018 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:35 dist-test-slave-jlzn krb5kdc[1223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:35 dist-test-slave-jlzn krb5kdc[1223](info): set up 2 sockets
Aug 08 15:07:35 dist-test-slave-jlzn krb5kdc[1223](info): commencing operation
krb5kdc: starting...
W20260808 15:07:36.988451 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:37 dist-test-slave-jlzn krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201657, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:37.227483 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:37.547914 1248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:37.548202 1248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:37.548338 1248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:37.555215 1248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:37.555322 1248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:37.555397 1248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:37.555478 1248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:37.555538 1248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:37.564698 1248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:37.565843 1248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:37.567039 1248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:37.577999 1253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:37.577962 1256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:37.578202 1258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:37.578624 1248 server_base.cc:1034] running on GCE node
I20260808 15:07:37.579393 1248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:37.580680 1248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:37.581888 1248 hybrid_clock.cc:648] HybridClock initialized: now 1786201657581833 us; error 40 us; skew 500 ppm
I20260808 15:07:37.582211 1248 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:37 dist-test-slave-jlzn krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201657, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:37.585218 1248 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:37.586530 1248 webserver.cc:460] Webserver started at http://127.0.0.1:41881/ using document root <none> and password file <none>
I20260808 15:07:37.587158 1248 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:37.587276 1248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:37.587575 1248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:37.590427 1248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "f4ed0a35e38c4e8d8235f30f341e789e"
format_stamp: "Formatted at 2026-08-08 15:07:37 on dist-test-slave-jlzn"
server_key: "40e471b3a8ffda748aaa7e5c2d15ccf3"
server_key_iv: "bcc465be314637d3c55b3498fe85b52d"
server_key_version: "encryptionkey@0"
I20260808 15:07:37.591159 1248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "f4ed0a35e38c4e8d8235f30f341e789e"
format_stamp: "Formatted at 2026-08-08 15:07:37 on dist-test-slave-jlzn"
server_key: "40e471b3a8ffda748aaa7e5c2d15ccf3"
server_key_iv: "bcc465be314637d3c55b3498fe85b52d"
server_key_version: "encryptionkey@0"
I20260808 15:07:37.595947 1248 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 15:07:37.599139 1266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:37.600376 1248 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:07:37.600538 1248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "f4ed0a35e38c4e8d8235f30f341e789e"
format_stamp: "Formatted at 2026-08-08 15:07:37 on dist-test-slave-jlzn"
server_key: "40e471b3a8ffda748aaa7e5c2d15ccf3"
server_key_iv: "bcc465be314637d3c55b3498fe85b52d"
server_key_version: "encryptionkey@0"
I20260808 15:07:37.600705 1248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:37.623296 1248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:37.626945 1248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:37.627205 1248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:37.643981 1248 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44725
I20260808 15:07:37.644011 1343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44725 every 8 connection(s)
I20260808 15:07:37.645251 1248 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:37.648458 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 1248
I20260808 15:07:37.648618 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:37.648680 1344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:37.649070 2771 external_mini_cluster.cc:1428] Setting key 6ace5b9982d5f05ea0805476073fe6d9
I20260808 15:07:37.662662 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:37.650694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47879 (local address 127.0.0.1:44725)
0808 15:07:37.651534 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:37.651548 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:37.651599 (+ 51us) server_negotiation.cc:407] Connection header received
0808 15:07:37.653371 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:37.653399 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:37.653866 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:37.654266 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:37.654761 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:37.656719 (+ 1958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:37.657962 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:37.658931 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:37.659108 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:37.660147 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:37.660158 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:37.660174 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:37.660868 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:37.660963 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:37.660999 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:37.661119 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:37.661259 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 15:07:37.661490 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":326,"threads_started":1}
I20260808 15:07:37.663208 1344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e: Bootstrap starting.
I20260808 15:07:37.665966 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:37.666195 1344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:37.667304 1344 log.cc:824] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:37.670318 1344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e: No bootstrap required, opened a new log
I20260808 15:07:37.676795 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:37.670703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40037 (local address 127.0.0.1:44725)
0808 15:07:37.670870 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:37.670875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:37.671251 (+ 376us) server_negotiation.cc:407] Connection header received
0808 15:07:37.671385 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:37.671390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:37.671457 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:37.671574 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:37.672061 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:37.673115 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:37.674385 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:37.675097 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:37.675220 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:37.676137 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:37.676143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:37.676147 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:37.676170 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:37.676216 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:37.676235 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:37.676320 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:37.676397 (+ 77us) server_negotiation.cc:299] Negotiation successful
0808 15:07:37.676480 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:07:37.678905 1344 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed0a35e38c4e8d8235f30f341e789e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } }
I20260808 15:07:37.679215 1344 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:37.679298 1344 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4ed0a35e38c4e8d8235f30f341e789e, State: Initialized, Role: FOLLOWER
I20260808 15:07:37.679739 1344 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed0a35e38c4e8d8235f30f341e789e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } }
I20260808 15:07:37.680020 1344 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:37.680120 1344 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:37.680261 1344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:37.682348 1344 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed0a35e38c4e8d8235f30f341e789e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } }
I20260808 15:07:37.682768 1344 leader_election.cc:302] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4ed0a35e38c4e8d8235f30f341e789e; no voters:
I20260808 15:07:37.683279 1344 leader_election.cc:288] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:37.683399 1361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:37.684008 1361 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [term 1 LEADER]: Becoming Leader. State: Replica: f4ed0a35e38c4e8d8235f30f341e789e, State: Running, Role: LEADER
I20260808 15:07:37.684495 1361 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed0a35e38c4e8d8235f30f341e789e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } }
I20260808 15:07:37.685048 1344 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:37.692482 1363 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4ed0a35e38c4e8d8235f30f341e789e. Latest consensus state: current_term: 1 leader_uuid: "f4ed0a35e38c4e8d8235f30f341e789e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed0a35e38c4e8d8235f30f341e789e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } } }
I20260808 15:07:37.692865 1363 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:37.693378 1362 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4ed0a35e38c4e8d8235f30f341e789e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4ed0a35e38c4e8d8235f30f341e789e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44725 } } }
I20260808 15:07:37.693655 1362 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e [sys.catalog]: This master's current role is: LEADER
W20260808 15:07:37.699051 1384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:07:37.699167 1384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:07:37.699301 1386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:37.701376 1386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:37.709161 1386 catalog_manager.cc:1347] Generated new cluster ID: 0a8787b0dad140a6bfe35f8b5ff53856
I20260808 15:07:37.709254 1386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:37.742439 1386 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:37.743719 1386 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:37.755511 1386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4ed0a35e38c4e8d8235f30f341e789e: Generated new TSK 0
I20260808 15:07:37.756332 1386 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:37.775447 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1248
W20260808 15:07:37.776542 1339 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47879
W20260808 15:07:37.777165 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1248: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:37.777316 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 1248
2026-08-08T15:07:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1981 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:37 dist-test-slave-jlzn krb5kdc[1396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:37 dist-test-slave-jlzn krb5kdc[1396](info): set up 2 sockets
Aug 08 15:07:37 dist-test-slave-jlzn krb5kdc[1396](info): commencing operation
krb5kdc: starting...
W20260808 15:07:38.963956 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.006s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:39 dist-test-slave-jlzn krb5kdc[1396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:39.204102 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39499
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39499
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:39.517696 1412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:39.517964 1412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:39.518077 1412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:39.525123 1412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:39.525223 1412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:39.525309 1412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:39.525385 1412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:39.525442 1412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:39.534921 1412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39499
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:39.536078 1412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:39.537271 1412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:39.548487 1418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:39.548615 1421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:39.548523 1419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:39.549007 1412 server_base.cc:1034] running on GCE node
I20260808 15:07:39.549917 1412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:39.551201 1412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:39.552408 1412 hybrid_clock.cc:648] HybridClock initialized: now 1786201659552359 us; error 35 us; skew 500 ppm
I20260808 15:07:39.552755 1412 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:39 dist-test-slave-jlzn krb5kdc[1396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201659, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:39.555702 1412 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:39.557049 1412 webserver.cc:460] Webserver started at http://127.0.0.1:45277/ using document root <none> and password file <none>
I20260808 15:07:39.557675 1412 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:39.557794 1412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:39.558070 1412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:39.560881 1412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "72f0b40db23b4468977529c4d010549a"
format_stamp: "Formatted at 2026-08-08 15:07:39 on dist-test-slave-jlzn"
server_key: "a271a83f7a17421d782e3a83a53dbbde"
server_key_iv: "fc150205491c3d092e9b4c3f53e7fa1c"
server_key_version: "encryptionkey@0"
I20260808 15:07:39.561628 1412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "72f0b40db23b4468977529c4d010549a"
format_stamp: "Formatted at 2026-08-08 15:07:39 on dist-test-slave-jlzn"
server_key: "a271a83f7a17421d782e3a83a53dbbde"
server_key_iv: "fc150205491c3d092e9b4c3f53e7fa1c"
server_key_version: "encryptionkey@0"
I20260808 15:07:39.566419 1412 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 15:07:39.569631 1435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:39.570788 1412 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:07:39.570946 1412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "72f0b40db23b4468977529c4d010549a"
format_stamp: "Formatted at 2026-08-08 15:07:39 on dist-test-slave-jlzn"
server_key: "a271a83f7a17421d782e3a83a53dbbde"
server_key_iv: "fc150205491c3d092e9b4c3f53e7fa1c"
server_key_version: "encryptionkey@0"
I20260808 15:07:39.571112 1412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:39.576429 1412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:39.579938 1412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:39.580191 1412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:39.596414 1412 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39499
I20260808 15:07:39.596467 1507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39499 every 8 connection(s)
I20260808 15:07:39.597592 1412 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:39.600965 1508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:39.603955 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 1412
I20260808 15:07:39.604137 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:39.604712 2771 external_mini_cluster.cc:1428] Setting key 885b8215503d6837520410a98f1791f4
I20260808 15:07:39.612008 1511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:39.606414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60449 (local address 127.0.0.1:39499)
0808 15:07:39.607161 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:39.607172 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:39.607208 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:07:39.608844 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:39.608867 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:39.609186 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:39.609524 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:39.610521 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:39.610527 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:39.610539 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:39.610554 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:39.610613 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:39.610633 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:39.610838 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:39.610970 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 15:07:39.611129 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":342,"threads_started":1}
I20260808 15:07:39.612344 1508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a: Bootstrap starting.
I20260808 15:07:39.615573 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:39.615797 1508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:39.616811 1508 log.cc:824] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:39.619441 1508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a: No bootstrap required, opened a new log
I20260808 15:07:39.622247 1511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:39.619927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49855 (local address 127.0.0.1:39499)
0808 15:07:39.620062 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:39.620067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:39.620482 (+ 415us) server_negotiation.cc:407] Connection header received
0808 15:07:39.620622 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:39.620627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:39.620686 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:39.620764 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:39.621619 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:39.621625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:39.621630 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:39.621646 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:39.621687 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:39.621705 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:39.621804 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:39.621908 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 15:07:39.621965 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 15:07:39.627326 1508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f0b40db23b4468977529c4d010549a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } }
I20260808 15:07:39.627660 1508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:39.627748 1508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72f0b40db23b4468977529c4d010549a, State: Initialized, Role: FOLLOWER
I20260808 15:07:39.628376 1508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f0b40db23b4468977529c4d010549a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } }
I20260808 15:07:39.628679 1508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:39.628825 1508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:39.628955 1508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:39.630900 1508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f0b40db23b4468977529c4d010549a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } }
I20260808 15:07:39.631309 1508 leader_election.cc:302] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72f0b40db23b4468977529c4d010549a; no voters:
I20260808 15:07:39.631783 1508 leader_election.cc:288] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:39.631944 1522 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:39.632555 1522 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [term 1 LEADER]: Becoming Leader. State: Replica: 72f0b40db23b4468977529c4d010549a, State: Running, Role: LEADER
I20260808 15:07:39.633087 1522 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f0b40db23b4468977529c4d010549a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } }
I20260808 15:07:39.633656 1508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:39.635604 1523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72f0b40db23b4468977529c4d010549a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f0b40db23b4468977529c4d010549a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } } }
I20260808 15:07:39.636159 1523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:39.635612 1524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72f0b40db23b4468977529c4d010549a. Latest consensus state: current_term: 1 leader_uuid: "72f0b40db23b4468977529c4d010549a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72f0b40db23b4468977529c4d010549a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39499 } } }
I20260808 15:07:39.636672 1524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:39.637115 1531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:39.642638 1531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:39.651230 1531 catalog_manager.cc:1347] Generated new cluster ID: 944946b064fe494d885fbfced096439c
I20260808 15:07:39.651335 1531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:39.684514 1531 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:39.684891 1531 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:39.692309 1531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72f0b40db23b4468977529c4d010549a: Generated new TSK 0
I20260808 15:07:39.692998 1531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:39.706321 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1412
W20260808 15:07:39.707288 1500 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:60449
W20260808 15:07:39.707902 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1412: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:39.708022 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 1412
2026-08-08T15:07:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1930 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:39 dist-test-slave-jlzn krb5kdc[1550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:39 dist-test-slave-jlzn krb5kdc[1550](info): set up 2 sockets
Aug 08 15:07:39 dist-test-slave-jlzn krb5kdc[1550](info): commencing operation
krb5kdc: starting...
W20260808 15:07:40.906013 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:41 dist-test-slave-jlzn krb5kdc[1550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:41.150938 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33117
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:41.473102 1566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:41.473371 1566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:41.473460 1566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:41.480341 1566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:41.480448 1566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:41.480500 1566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:41.480566 1566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:41.480610 1566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:41.490162 1566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:41.491362 1566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:41.492726 1566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:41.503826 1573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:41.503875 1578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:41.503882 1575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:41.504602 1566 server_base.cc:1034] running on GCE node
I20260808 15:07:41.505326 1566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:41.506606 1566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:41.507822 1566 hybrid_clock.cc:648] HybridClock initialized: now 1786201661507755 us; error 43 us; skew 500 ppm
I20260808 15:07:41.508157 1566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:41 dist-test-slave-jlzn krb5kdc[1550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201661, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:41.515058 1566 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:41.516422 1566 webserver.cc:460] Webserver started at http://127.0.0.1:38901/ using document root <none> and password file <none>
I20260808 15:07:41.517056 1566 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:41.517172 1566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:41.517449 1566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:41.520340 1566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "413a61d236a14dbea4f39b5d5d88e676"
format_stamp: "Formatted at 2026-08-08 15:07:41 on dist-test-slave-jlzn"
server_key: "61239f44c2acd845c3927f3a20428745"
server_key_iv: "53975c64fa85923e5f42bebc3a837163"
server_key_version: "encryptionkey@0"
I20260808 15:07:41.521055 1566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "413a61d236a14dbea4f39b5d5d88e676"
format_stamp: "Formatted at 2026-08-08 15:07:41 on dist-test-slave-jlzn"
server_key: "61239f44c2acd845c3927f3a20428745"
server_key_iv: "53975c64fa85923e5f42bebc3a837163"
server_key_version: "encryptionkey@0"
I20260808 15:07:41.525833 1566 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:07:41.529098 1587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:41.530296 1566 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:07:41.530455 1566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "413a61d236a14dbea4f39b5d5d88e676"
format_stamp: "Formatted at 2026-08-08 15:07:41 on dist-test-slave-jlzn"
server_key: "61239f44c2acd845c3927f3a20428745"
server_key_iv: "53975c64fa85923e5f42bebc3a837163"
server_key_version: "encryptionkey@0"
I20260808 15:07:41.530620 1566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:41.544129 1566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:41.547722 1566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:41.547997 1566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:41.564481 1566 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33117
I20260808 15:07:41.564520 1654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33117 every 8 connection(s)
I20260808 15:07:41.565747 1566 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:41.569198 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:41.573637 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 1566
I20260808 15:07:41.573815 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:41.574378 2771 external_mini_cluster.cc:1428] Setting key 4b09b56ee886f26fe9b855100a68ad6f
I20260808 15:07:41.580183 1657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676: Bootstrap starting.
I20260808 15:07:41.583603 1657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:41.584769 1657 log.cc:824] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:41.587874 1657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676: No bootstrap required, opened a new log
Aug 08 15:07:41 dist-test-slave-jlzn krb5kdc[1550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201661, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 15:07:41.595588 1657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413a61d236a14dbea4f39b5d5d88e676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } }
I20260808 15:07:41.596040 1657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:41.596159 1657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413a61d236a14dbea4f39b5d5d88e676, State: Initialized, Role: FOLLOWER
I20260808 15:07:41.596848 1662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:41.576091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:47505 (local address 127.0.0.1:33117)
0808 15:07:41.576810 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:41.576821 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:41.576867 (+ 46us) server_negotiation.cc:407] Connection header received
0808 15:07:41.578742 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:41.578763 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:41.579110 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:41.579402 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:41.581262 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:41.582531 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:41.584302 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:41.584948 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:41.585072 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:41.589667 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:41.589675 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:41.589691 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:41.589725 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:41.592746 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:41.593532 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:41.593538 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:41.593543 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:41.593704 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:41.594138 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:41.594142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:41.594144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:41.594660 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:41.594968 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:41.595428 (+ 460us) server_negotiation.cc:299] Negotiation successful
0808 15:07:41.595715 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":320,"threads_started":1}
I20260808 15:07:41.596800 1657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413a61d236a14dbea4f39b5d5d88e676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } }
I20260808 15:07:41.597123 1657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:41.597295 1657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:41.597499 1657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:41.600284 1657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413a61d236a14dbea4f39b5d5d88e676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } }
I20260808 15:07:41.600786 1657 leader_election.cc:302] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 413a61d236a14dbea4f39b5d5d88e676; no voters:
I20260808 15:07:41.600859 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:41.601377 1657 leader_election.cc:288] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:41.601500 1666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:41.603497 1657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:41.608162 1666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [term 1 LEADER]: Becoming Leader. State: Replica: 413a61d236a14dbea4f39b5d5d88e676, State: Running, Role: LEADER
I20260808 15:07:41.608774 1666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413a61d236a14dbea4f39b5d5d88e676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } }
I20260808 15:07:41.611268 1667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "413a61d236a14dbea4f39b5d5d88e676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413a61d236a14dbea4f39b5d5d88e676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } } }
I20260808 15:07:41.611652 1667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:41.615746 1668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 413a61d236a14dbea4f39b5d5d88e676. Latest consensus state: current_term: 1 leader_uuid: "413a61d236a14dbea4f39b5d5d88e676" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413a61d236a14dbea4f39b5d5d88e676" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33117 } } }
I20260808 15:07:41.616078 1668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676 [sys.catalog]: This master's current role is: LEADER
W20260808 15:07:41.618456 1694 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:07:41.618572 1694 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:07:41.618687 1690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:41.620666 1690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:41.626959 1662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:41.606316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:60615 (local address 127.0.0.1:33117)
0808 15:07:41.606574 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:41.606578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:41.610784 (+ 4206us) server_negotiation.cc:407] Connection header received
0808 15:07:41.610970 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:41.610975 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:41.611062 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:41.611162 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:41.613038 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:41.613956 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:41.615819 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:41.616527 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:41.616654 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:41.619655 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:41.619662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:41.619667 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:41.619705 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:41.623603 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:41.624547 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:41.624561 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:41.624565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:41.624754 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:41.625298 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:41.625307 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:41.625310 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:41.625733 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:41.626021 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:41.626425 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 15:07:41.626620 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260808 15:07:41.630201 1690 catalog_manager.cc:1347] Generated new cluster ID: 682f4534d3364ceb890985df91f26be0
I20260808 15:07:41.630291 1690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:41.642037 1690 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:41.643216 1690 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:41.656728 1690 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 413a61d236a14dbea4f39b5d5d88e676: Generated new TSK 0
I20260808 15:07:41.657455 1690 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:41.663005 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1566
I20260808 15:07:41.664224 1650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:41.754189 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 1566
W20260808 15:07:41.776304 1548 connection.cc:535] client connection to 127.0.0.1:33117 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33117): Connection reset by peer (error 104)
2026-08-08T15:07:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2046 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:41 dist-test-slave-jlzn krb5kdc[1701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:41 dist-test-slave-jlzn krb5kdc[1701](info): set up 2 sockets
Aug 08 15:07:41 dist-test-slave-jlzn krb5kdc[1701](info): commencing operation
krb5kdc: starting...
W20260808 15:07:42.958010 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:43 dist-test-slave-jlzn krb5kdc[1701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201663, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:43.202185 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37685
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:43.524075 1719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:43.524324 1719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:43.524448 1719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:43.531318 1719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:43.531436 1719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:43.531531 1719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:43.531625 1719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:43.531687 1719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:43.540894 1719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:43.542017 1719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:43.543267 1719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:43.554353 1732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:43.554306 1734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:43.554389 1738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:43.555331 1719 server_base.cc:1034] running on GCE node
I20260808 15:07:43.556061 1719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:43.557291 1719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:43.558476 1719 hybrid_clock.cc:648] HybridClock initialized: now 1786201663558432 us; error 41 us; skew 500 ppm
I20260808 15:07:43.558787 1719 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:43 dist-test-slave-jlzn krb5kdc[1701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201663, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:43.561717 1719 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:43.562985 1719 webserver.cc:460] Webserver started at http://127.0.0.1:45135/ using document root <none> and password file <none>
I20260808 15:07:43.563565 1719 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:43.563669 1719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:43.563961 1719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:43.566666 1719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "f46901b1bf744bcba1d40305e1284543"
format_stamp: "Formatted at 2026-08-08 15:07:43 on dist-test-slave-jlzn"
server_key: "b9676196b7d5d59761bd72ce80f7effa"
server_key_iv: "2f7046b5d98a33c2cb1366d343fb09fa"
server_key_version: "encryptionkey@0"
I20260808 15:07:43.567334 1719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "f46901b1bf744bcba1d40305e1284543"
format_stamp: "Formatted at 2026-08-08 15:07:43 on dist-test-slave-jlzn"
server_key: "b9676196b7d5d59761bd72ce80f7effa"
server_key_iv: "2f7046b5d98a33c2cb1366d343fb09fa"
server_key_version: "encryptionkey@0"
I20260808 15:07:43.572209 1719 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 15:07:43.575541 1745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:43.576879 1719 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 15:07:43.577042 1719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "f46901b1bf744bcba1d40305e1284543"
format_stamp: "Formatted at 2026-08-08 15:07:43 on dist-test-slave-jlzn"
server_key: "b9676196b7d5d59761bd72ce80f7effa"
server_key_iv: "2f7046b5d98a33c2cb1366d343fb09fa"
server_key_version: "encryptionkey@0"
I20260808 15:07:43.577225 1719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:43.599643 1719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:43.603420 1719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:43.603685 1719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:43.619833 1719 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37685
I20260808 15:07:43.619861 1817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37685 every 8 connection(s)
I20260808 15:07:43.621120 1719 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:43.624696 1818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:43.626219 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 1719
I20260808 15:07:43.626379 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:43.626827 2771 external_mini_cluster.cc:1428] Setting key 934d4bbc9dffffbd4b9758e4aaddc5d0
I20260808 15:07:43.637702 1818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543: Bootstrap starting.
I20260808 15:07:43.639328 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:43.628498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:45723 (local address 127.0.0.1:37685)
0808 15:07:43.629262 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:43.629273 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:43.629322 (+ 49us) server_negotiation.cc:407] Connection header received
0808 15:07:43.630911 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:43.630937 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:43.631331 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:43.631640 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:43.632194 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:43.633419 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:43.635201 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:43.635897 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:43.636016 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:43.637237 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:43.637244 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:43.637256 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:43.637757 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:43.637836 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:43.637867 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:43.637981 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:43.638247 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 15:07:43.638409 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":320,"threads_started":1}
I20260808 15:07:43.641003 1818 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:43.642127 1818 log.cc:824] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:43.643199 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:43.645478 1818 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543: No bootstrap required, opened a new log
I20260808 15:07:43.655210 1818 raft_consensus.cc:348] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46901b1bf744bcba1d40305e1284543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37685 } }
I20260808 15:07:43.655650 1818 raft_consensus.cc:374] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:43.655759 1818 raft_consensus.cc:729] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f46901b1bf744bcba1d40305e1284543, State: Initialized, Role: FOLLOWER
I20260808 15:07:43.656406 1818 consensus_queue.cc:260] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46901b1bf744bcba1d40305e1284543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37685 } }
I20260808 15:07:43.656688 1818 raft_consensus.cc:388] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:43.656827 1818 raft_consensus.cc:482] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:43.656976 1818 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:43.659022 1818 raft_consensus.cc:504] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46901b1bf744bcba1d40305e1284543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37685 } }
I20260808 15:07:43.659476 1818 leader_election.cc:302] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f46901b1bf744bcba1d40305e1284543; no voters:
I20260808 15:07:43.660851 1818 leader_election.cc:288] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:43.660944 1835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:43.663028 1818 sys_catalog.cc:564] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:43.663928 1835 raft_consensus.cc:686] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [term 1 LEADER]: Becoming Leader. State: Replica: f46901b1bf744bcba1d40305e1284543, State: Running, Role: LEADER
I20260808 15:07:43.664609 1835 consensus_queue.cc:237] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46901b1bf744bcba1d40305e1284543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37685 } }
I20260808 15:07:43.667618 1841 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f46901b1bf744bcba1d40305e1284543. Latest consensus state: current_term: 1 leader_uuid: "f46901b1bf744bcba1d40305e1284543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46901b1bf744bcba1d40305e1284543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37685 } } }
I20260808 15:07:43.668020 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:43.649141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:58127 (local address 127.0.0.1:37685)
0808 15:07:43.649359 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:43.649365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:43.660367 (+ 11002us) server_negotiation.cc:407] Connection header received
0808 15:07:43.660550 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:43.660553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:43.660618 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:43.660713 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:43.661336 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:43.662320 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:43.663689 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:43.664363 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:43.664499 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:43.666925 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:43.666932 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:43.666936 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:43.666958 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:43.667020 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:43.667041 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:43.667146 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:43.667612 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 15:07:43.667667 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 15:07:43.668051 1841 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:43.668838 1839 sys_catalog.cc:455] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f46901b1bf744bcba1d40305e1284543" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f46901b1bf744bcba1d40305e1284543" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37685 } } }
I20260808 15:07:43.669112 1839 sys_catalog.cc:458] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:43.673225 1851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:43.678162 1851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:43.686868 1851 catalog_manager.cc:1347] Generated new cluster ID: b7ecd59eece34c6a9fc790c2e6e46ea8
I20260808 15:07:43.686955 1851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:43.714639 1851 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:43.715960 1851 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:43.721820 1851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f46901b1bf744bcba1d40305e1284543: Generated new TSK 0
I20260808 15:07:43.722630 1851 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:43.736192 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1719
W20260808 15:07:43.737253 1813 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:45723
W20260808 15:07:43.737954 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1719: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:43.738087 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 1719
W20260808 15:07:43.760973 1699 connection.cc:535] client connection to 127.0.0.1:37685 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37685): Connection reset by peer (error 104)
2026-08-08T15:07:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1984 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:43 dist-test-slave-jlzn krb5kdc[1870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:43 dist-test-slave-jlzn krb5kdc[1870](info): set up 2 sockets
Aug 08 15:07:43 dist-test-slave-jlzn krb5kdc[1870](info): commencing operation
krb5kdc: starting...
W20260808 15:07:44.932749 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:45 dist-test-slave-jlzn krb5kdc[1870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201665, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:45.181185 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41573
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:45.501086 1886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:45.501358 1886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:45.501456 1886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:45.508385 1886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:45.508481 1886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:45.508533 1886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:45.508605 1886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:45.508651 1886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:45.517925 1886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41573
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:45.519052 1886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:45.520313 1886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:45.531184 1893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:45.531225 1894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:45.531270 1896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:45.532315 1886 server_base.cc:1034] running on GCE node
I20260808 15:07:45.533000 1886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:45.534268 1886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:45.535470 1886 hybrid_clock.cc:648] HybridClock initialized: now 1786201665535428 us; error 36 us; skew 500 ppm
I20260808 15:07:45.535784 1886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:45 dist-test-slave-jlzn krb5kdc[1870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201665, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:45.538754 1886 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:45.540113 1886 webserver.cc:460] Webserver started at http://127.0.0.1:41911/ using document root <none> and password file <none>
I20260808 15:07:45.540768 1886 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:45.540887 1886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:45.541167 1886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:45.544051 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "1566b609abc742dabef6da8c6b5664dd"
format_stamp: "Formatted at 2026-08-08 15:07:45 on dist-test-slave-jlzn"
server_key: "4bc34afb923999e7413af6f01a04debf"
server_key_iv: "048c2cf60c05c3d062f44cbb96b3d417"
server_key_version: "encryptionkey@0"
I20260808 15:07:45.544785 1886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "1566b609abc742dabef6da8c6b5664dd"
format_stamp: "Formatted at 2026-08-08 15:07:45 on dist-test-slave-jlzn"
server_key: "4bc34afb923999e7413af6f01a04debf"
server_key_iv: "048c2cf60c05c3d062f44cbb96b3d417"
server_key_version: "encryptionkey@0"
I20260808 15:07:45.549540 1886 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:07:45.552697 1907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:45.553835 1886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 15:07:45.553984 1886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "1566b609abc742dabef6da8c6b5664dd"
format_stamp: "Formatted at 2026-08-08 15:07:45 on dist-test-slave-jlzn"
server_key: "4bc34afb923999e7413af6f01a04debf"
server_key_iv: "048c2cf60c05c3d062f44cbb96b3d417"
server_key_version: "encryptionkey@0"
I20260808 15:07:45.554162 1886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:45.559937 1886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:45.564015 1886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:45.564284 1886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:45.580631 1886 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41573
I20260808 15:07:45.580686 1987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41573 every 8 connection(s)
I20260808 15:07:45.581869 1886 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:45.585287 1988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:45.591354 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 1886
I20260808 15:07:45.591555 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:45.592142 2771 external_mini_cluster.cc:1428] Setting key 61e960d1b813b3cd6b10dcda302ef495
I20260808 15:07:45.595897 1988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd: Bootstrap starting.
I20260808 15:07:45.599064 1988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:45.599738 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:45.593942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:54671 (local address 127.0.0.1:41573)
0808 15:07:45.594700 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:45.594711 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:45.594749 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:07:45.596506 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:45.596530 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:45.596847 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:45.597213 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:45.598196 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:45.598202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:45.598214 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:45.598229 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:45.598285 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:45.598304 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:45.598470 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:45.598603 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 15:07:45.598783 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":305,"threads_started":1}
I20260808 15:07:45.600207 1988 log.cc:824] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:45.603181 1988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd: No bootstrap required, opened a new log
I20260808 15:07:45.603363 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:45.610741 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:45.608470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:60861 (local address 127.0.0.1:41573)
0808 15:07:45.608634 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:45.608640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:45.608935 (+ 295us) server_negotiation.cc:407] Connection header received
0808 15:07:45.609066 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:45.609071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:45.609135 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:45.609245 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:45.610139 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:45.610145 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:45.610149 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:45.610166 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:45.610210 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:45.610228 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:45.610354 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:45.610400 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 15:07:45.610457 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:07:45.612802 1988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1566b609abc742dabef6da8c6b5664dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41573 } }
I20260808 15:07:45.613133 1988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:45.613206 1988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1566b609abc742dabef6da8c6b5664dd, State: Initialized, Role: FOLLOWER
I20260808 15:07:45.613672 1988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1566b609abc742dabef6da8c6b5664dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41573 } }
I20260808 15:07:45.613878 1988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:45.613979 1988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:45.614110 1988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:45.616156 1988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1566b609abc742dabef6da8c6b5664dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41573 } }
I20260808 15:07:45.616575 1988 leader_election.cc:302] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1566b609abc742dabef6da8c6b5664dd; no voters:
I20260808 15:07:45.617234 1988 leader_election.cc:288] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:45.617359 2004 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:45.618085 2004 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [term 1 LEADER]: Becoming Leader. State: Replica: 1566b609abc742dabef6da8c6b5664dd, State: Running, Role: LEADER
I20260808 15:07:45.618609 2004 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1566b609abc742dabef6da8c6b5664dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41573 } }
I20260808 15:07:45.619360 1988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:45.621630 2008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1566b609abc742dabef6da8c6b5664dd. Latest consensus state: current_term: 1 leader_uuid: "1566b609abc742dabef6da8c6b5664dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1566b609abc742dabef6da8c6b5664dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41573 } } }
I20260808 15:07:45.621702 2006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1566b609abc742dabef6da8c6b5664dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1566b609abc742dabef6da8c6b5664dd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41573 } } }
I20260808 15:07:45.622107 2006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:45.622107 2008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:45.629248 2016 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:45.633898 2016 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:45.641839 2016 catalog_manager.cc:1347] Generated new cluster ID: c60aba0b61cd4018a6c18f3bd0fed847
I20260808 15:07:45.641947 2016 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:45.667632 2016 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:45.668118 2016 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:45.674841 2016 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1566b609abc742dabef6da8c6b5664dd: Generated new TSK 0
I20260808 15:07:45.675594 2016 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:45.689355 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1886
W20260808 15:07:45.690335 1982 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:54671
W20260808 15:07:45.690963 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 1886: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:45.691088 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 1886
2026-08-08T15:07:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1955 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:45 dist-test-slave-jlzn krb5kdc[2030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:45 dist-test-slave-jlzn krb5kdc[2030](info): set up 2 sockets
Aug 08 15:07:45 dist-test-slave-jlzn krb5kdc[2030](info): commencing operation
krb5kdc: starting...
W20260808 15:07:46.887202 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:47 dist-test-slave-jlzn krb5kdc[2030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:47.148939 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:47.471308 2044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:47.471585 2044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:47.471709 2044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:47.478606 2044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:47.478713 2044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:47.478782 2044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:47.478863 2044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:47.478916 2044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:47.488363 2044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:47.489513 2044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:47.490752 2044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:47.501900 2055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:47.501952 2051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:47.501977 2052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:47.502890 2044 server_base.cc:1034] running on GCE node
I20260808 15:07:47.503671 2044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:47.504959 2044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:47.506170 2044 hybrid_clock.cc:648] HybridClock initialized: now 1786201667506115 us; error 39 us; skew 500 ppm
I20260808 15:07:47.506484 2044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:47 dist-test-slave-jlzn krb5kdc[2030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201667, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:47.514163 2044 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:47.515558 2044 webserver.cc:460] Webserver started at http://127.0.0.1:43333/ using document root <none> and password file <none>
I20260808 15:07:47.516259 2044 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:47.516378 2044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:47.516672 2044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:47.519522 2044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "13c5e6b5728842d18c3c7c07d9fb2181"
format_stamp: "Formatted at 2026-08-08 15:07:47 on dist-test-slave-jlzn"
server_key: "51456b4c0c8776499111008d64549565"
server_key_iv: "d62358ff6dd6c7ce57d511e07d4e352c"
server_key_version: "encryptionkey@0"
I20260808 15:07:47.520300 2044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "13c5e6b5728842d18c3c7c07d9fb2181"
format_stamp: "Formatted at 2026-08-08 15:07:47 on dist-test-slave-jlzn"
server_key: "51456b4c0c8776499111008d64549565"
server_key_iv: "d62358ff6dd6c7ce57d511e07d4e352c"
server_key_version: "encryptionkey@0"
I20260808 15:07:47.525053 2044 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 15:07:47.528231 2069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:47.529404 2044 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:07:47.529561 2044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "13c5e6b5728842d18c3c7c07d9fb2181"
format_stamp: "Formatted at 2026-08-08 15:07:47 on dist-test-slave-jlzn"
server_key: "51456b4c0c8776499111008d64549565"
server_key_iv: "d62358ff6dd6c7ce57d511e07d4e352c"
server_key_version: "encryptionkey@0"
I20260808 15:07:47.529727 2044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:47.542748 2044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:47.546233 2044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:47.546483 2044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:47.563464 2044 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44359
I20260808 15:07:47.563498 2152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44359 every 8 connection(s)
I20260808 15:07:47.564777 2044 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:47.568215 2154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:47.570336 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 2044
I20260808 15:07:47.570510 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:47.571027 2771 external_mini_cluster.cc:1428] Setting key 7b6f416626ad5c63bb3b2aa74e7ebf4f
I20260808 15:07:47.579936 2154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181: Bootstrap starting.
I20260808 15:07:47.583454 2154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:47.584717 2154 log.cc:824] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:07:47 dist-test-slave-jlzn krb5kdc[2030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 15:07:47.587491 2154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181: No bootstrap required, opened a new log
I20260808 15:07:47.593619 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:47.572770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:51769 (local address 127.0.0.1:44359)
0808 15:07:47.573512 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:47.573525 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:47.573566 (+ 41us) server_negotiation.cc:407] Connection header received
0808 15:07:47.575226 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:47.575249 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:47.575602 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:47.575956 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:47.577633 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:47.578932 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:47.580666 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:47.581315 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:47.581469 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:47.586356 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:47.586363 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:47.586377 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:47.586411 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:47.589373 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:47.590199 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:47.590205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:47.590214 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:47.590386 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:47.590850 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:47.590854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:47.590856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:47.591402 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:47.591764 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:47.592318 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 15:07:47.592624 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":326,"threads_started":1}
I20260808 15:07:47.595067 2154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c5e6b5728842d18c3c7c07d9fb2181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260808 15:07:47.595407 2154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:47.595506 2154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13c5e6b5728842d18c3c7c07d9fb2181, State: Initialized, Role: FOLLOWER
I20260808 15:07:47.596134 2154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c5e6b5728842d18c3c7c07d9fb2181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260808 15:07:47.596360 2154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:47.596487 2154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:47.596644 2154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:47.597280 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:47.598829 2154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c5e6b5728842d18c3c7c07d9fb2181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260808 15:07:47.599269 2154 leader_election.cc:302] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13c5e6b5728842d18c3c7c07d9fb2181; no voters:
I20260808 15:07:47.599941 2154 leader_election.cc:288] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:47.600041 2161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:47.601455 2161 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [term 1 LEADER]: Becoming Leader. State: Replica: 13c5e6b5728842d18c3c7c07d9fb2181, State: Running, Role: LEADER
I20260808 15:07:47.602115 2154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:47.602077 2161 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c5e6b5728842d18c3c7c07d9fb2181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } }
I20260808 15:07:47.605147 2164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13c5e6b5728842d18c3c7c07d9fb2181" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c5e6b5728842d18c3c7c07d9fb2181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } } }
I20260808 15:07:47.605489 2164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:47.610917 2168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13c5e6b5728842d18c3c7c07d9fb2181. Latest consensus state: current_term: 1 leader_uuid: "13c5e6b5728842d18c3c7c07d9fb2181" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13c5e6b5728842d18c3c7c07d9fb2181" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44359 } } }
I20260808 15:07:47.611158 2168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181 [sys.catalog]: This master's current role is: LEADER
W20260808 15:07:47.616146 2195 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:07:47.616282 2195 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:07:47.616457 2179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:47.618745 2179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:47.624467 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:47.602379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:41323 (local address 127.0.0.1:44359)
0808 15:07:47.602521 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:47.602525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:47.609308 (+ 6783us) server_negotiation.cc:407] Connection header received
0808 15:07:47.609471 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:47.609476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:47.609557 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:47.609653 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:47.611756 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:47.612746 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:47.614015 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:47.614460 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:47.614569 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:47.617337 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:47.617347 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:47.617352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:47.617388 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:47.621115 (+ 3727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:47.622136 (+ 1021us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:47.622142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:47.622149 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:47.622289 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:47.622817 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:47.622823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:47.622827 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:47.623182 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:47.623459 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:47.624022 (+ 563us) server_negotiation.cc:299] Negotiation successful
0808 15:07:47.624212 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 15:07:47.627912 2179 catalog_manager.cc:1347] Generated new cluster ID: 639bd600663e4c00872c0c6de6806a05
I20260808 15:07:47.628012 2179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:47.653555 2179 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:47.654744 2179 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:47.659930 2179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13c5e6b5728842d18c3c7c07d9fb2181: Generated new TSK 0
I20260808 15:07:47.660668 2179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:47.668486 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2044
I20260808 15:07:47.669607 2146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:47.757411 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 2044
W20260808 15:07:47.781139 2028 connection.cc:535] client connection to 127.0.0.1:44359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44359): Connection reset by peer (error 104)
2026-08-08T15:07:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2064 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:47 dist-test-slave-jlzn krb5kdc[2202](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:47 dist-test-slave-jlzn krb5kdc[2202](info): set up 2 sockets
Aug 08 15:07:47 dist-test-slave-jlzn krb5kdc[2202](info): commencing operation
krb5kdc: starting...
W20260808 15:07:48.951212 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.006s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:49 dist-test-slave-jlzn krb5kdc[2202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:49.200848 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41343
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:49.516660 2219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:49.516922 2219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:49.517055 2219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:49.523918 2219 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:49.524014 2219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:49.524091 2219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:49.524173 2219 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:49.524235 2219 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:49.533452 2219 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41343
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:49.534579 2219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:49.535779 2219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:49.546574 2226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:49.546612 2230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:49.546644 2228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:49.547605 2219 server_base.cc:1034] running on GCE node
I20260808 15:07:49.548310 2219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:49.549489 2219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:49.550675 2219 hybrid_clock.cc:648] HybridClock initialized: now 1786201669550628 us; error 43 us; skew 500 ppm
I20260808 15:07:49.550981 2219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:49 dist-test-slave-jlzn krb5kdc[2202](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201669, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:49.553977 2219 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:49.555292 2219 webserver.cc:460] Webserver started at http://127.0.0.1:44635/ using document root <none> and password file <none>
I20260808 15:07:49.555927 2219 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:49.556044 2219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:49.556321 2219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:49.559103 2219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "783e96dd241c4aea8bee3ccc6aaf2996"
format_stamp: "Formatted at 2026-08-08 15:07:49 on dist-test-slave-jlzn"
server_key: "5e200d854a20182a387b728f5ff2b9c4"
server_key_iv: "20933e5d467aa456716cf4195ece953e"
server_key_version: "encryptionkey@0"
I20260808 15:07:49.559851 2219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "783e96dd241c4aea8bee3ccc6aaf2996"
format_stamp: "Formatted at 2026-08-08 15:07:49 on dist-test-slave-jlzn"
server_key: "5e200d854a20182a387b728f5ff2b9c4"
server_key_iv: "20933e5d467aa456716cf4195ece953e"
server_key_version: "encryptionkey@0"
I20260808 15:07:49.564558 2219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 15:07:49.567667 2240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:49.568841 2219 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:07:49.569000 2219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "783e96dd241c4aea8bee3ccc6aaf2996"
format_stamp: "Formatted at 2026-08-08 15:07:49 on dist-test-slave-jlzn"
server_key: "5e200d854a20182a387b728f5ff2b9c4"
server_key_iv: "20933e5d467aa456716cf4195ece953e"
server_key_version: "encryptionkey@0"
I20260808 15:07:49.569164 2219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:49.591200 2219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:49.595386 2219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:49.595645 2219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:49.612267 2219 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41343
I20260808 15:07:49.612308 2315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41343 every 8 connection(s)
I20260808 15:07:49.613569 2219 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:49.616974 2316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:49.622974 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 2219
I20260808 15:07:49.623143 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:49.623549 2771 external_mini_cluster.cc:1428] Setting key 740a27af600a3200125158a575d893ee
I20260808 15:07:49.626997 2316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996: Bootstrap starting.
I20260808 15:07:49.630127 2316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:49.631342 2316 log.cc:824] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:49.634372 2316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996: No bootstrap required, opened a new log
I20260808 15:07:49.636466 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:49.625251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:37611 (local address 127.0.0.1:41343)
0808 15:07:49.625940 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:49.625950 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:49.625988 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:07:49.627607 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:49.627628 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:49.628016 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:49.628313 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:49.628896 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:49.630157 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:49.632060 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:49.632940 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:49.633062 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:49.634250 (+ 1188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:49.634257 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:49.634269 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:49.634800 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:49.634876 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:49.634907 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:49.635024 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:49.635303 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 15:07:49.635462 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":305,"threads_started":1}
I20260808 15:07:49.640108 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:49.642307 2316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783e96dd241c4aea8bee3ccc6aaf2996" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41343 } }
I20260808 15:07:49.642657 2316 raft_consensus.cc:374] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:49.642760 2316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 783e96dd241c4aea8bee3ccc6aaf2996, State: Initialized, Role: FOLLOWER
I20260808 15:07:49.643309 2316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783e96dd241c4aea8bee3ccc6aaf2996" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41343 } }
I20260808 15:07:49.643522 2316 raft_consensus.cc:388] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:49.643646 2316 raft_consensus.cc:482] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:49.643826 2316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:49.646608 2316 raft_consensus.cc:504] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783e96dd241c4aea8bee3ccc6aaf2996" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41343 } }
I20260808 15:07:49.647068 2316 leader_election.cc:302] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 783e96dd241c4aea8bee3ccc6aaf2996; no voters:
I20260808 15:07:49.647630 2316 leader_election.cc:288] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:49.647770 2331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:49.648861 2331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [term 1 LEADER]: Becoming Leader. State: Replica: 783e96dd241c4aea8bee3ccc6aaf2996, State: Running, Role: LEADER
I20260808 15:07:49.649578 2316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:49.649487 2331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783e96dd241c4aea8bee3ccc6aaf2996" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41343 } }
I20260808 15:07:49.651252 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:49.644308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:44339 (local address 127.0.0.1:41343)
0808 15:07:49.644483 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:49.644488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:49.644849 (+ 361us) server_negotiation.cc:407] Connection header received
0808 15:07:49.644991 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:49.644997 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:49.645071 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:49.645169 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:49.645612 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:49.646640 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:49.648174 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:49.648929 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:49.649036 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:49.650408 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:49.650415 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:49.650420 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:49.650439 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:49.650499 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:49.650519 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:49.650638 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:49.650833 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 15:07:49.650884 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:07:49.654603 2332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "783e96dd241c4aea8bee3ccc6aaf2996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783e96dd241c4aea8bee3ccc6aaf2996" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41343 } } }
I20260808 15:07:49.654997 2332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:49.655277 2334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 783e96dd241c4aea8bee3ccc6aaf2996. Latest consensus state: current_term: 1 leader_uuid: "783e96dd241c4aea8bee3ccc6aaf2996" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "783e96dd241c4aea8bee3ccc6aaf2996" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41343 } } }
I20260808 15:07:49.655604 2334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:49.656684 2349 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:49.660895 2349 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:49.668681 2349 catalog_manager.cc:1347] Generated new cluster ID: ad892f1ea3314a7b975810c0088e1d96
I20260808 15:07:49.668768 2349 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:49.693288 2349 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:49.694463 2349 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:49.702661 2349 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 783e96dd241c4aea8bee3ccc6aaf2996: Generated new TSK 0
I20260808 15:07:49.703418 2349 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:49.715965 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2219
W20260808 15:07:49.716930 2310 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:37611
W20260808 15:07:49.717590 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2219: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:49.717710 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 2219
W20260808 15:07:49.741243 2200 connection.cc:535] client connection to 127.0.0.1:41343 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41343): Connection reset by peer (error 104)
2026-08-08T15:07:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1959 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:49 dist-test-slave-jlzn krb5kdc[2359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:49 dist-test-slave-jlzn krb5kdc[2359](info): set up 2 sockets
Aug 08 15:07:49 dist-test-slave-jlzn krb5kdc[2359](info): commencing operation
krb5kdc: starting...
W20260808 15:07:50.910606 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.003s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:51 dist-test-slave-jlzn krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 15:07:51.163978 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44547
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:51.485189 2382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:51.485461 2382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:51.485615 2382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:51.492529 2382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:51.492660 2382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:51.492741 2382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:51.492825 2382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:51.492887 2382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:51.502215 2382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44547
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:51.503407 2382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:51.504724 2382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:51.515971 2393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:51.515938 2395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:51.515822 2392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:51.516737 2382 server_base.cc:1034] running on GCE node
I20260808 15:07:51.517418 2382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:51.518689 2382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:51.519902 2382 hybrid_clock.cc:648] HybridClock initialized: now 1786201671519853 us; error 33 us; skew 500 ppm
I20260808 15:07:51.520241 2382 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:51 dist-test-slave-jlzn krb5kdc[2359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201671, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:51.523206 2382 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 15:07:51.524605 2382 webserver.cc:460] Webserver started at http://127.0.0.1:45209/ using document root <none> and password file <none>
I20260808 15:07:51.525220 2382 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:51.525336 2382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:51.525620 2382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:51.528488 2382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "825ca75139a94784a573c2100562c733"
format_stamp: "Formatted at 2026-08-08 15:07:51 on dist-test-slave-jlzn"
server_key: "0666762d4924de6d7ad53082937a5642"
server_key_iv: "2c9e5237ac9c8a15cd121ef9695c5651"
server_key_version: "encryptionkey@0"
I20260808 15:07:51.529228 2382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "825ca75139a94784a573c2100562c733"
format_stamp: "Formatted at 2026-08-08 15:07:51 on dist-test-slave-jlzn"
server_key: "0666762d4924de6d7ad53082937a5642"
server_key_iv: "2c9e5237ac9c8a15cd121ef9695c5651"
server_key_version: "encryptionkey@0"
I20260808 15:07:51.534071 2382 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 15:07:51.537331 2404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:51.538501 2382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 15:07:51.538678 2382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "825ca75139a94784a573c2100562c733"
format_stamp: "Formatted at 2026-08-08 15:07:51 on dist-test-slave-jlzn"
server_key: "0666762d4924de6d7ad53082937a5642"
server_key_iv: "2c9e5237ac9c8a15cd121ef9695c5651"
server_key_version: "encryptionkey@0"
I20260808 15:07:51.538853 2382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:51.544353 2382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:51.548327 2382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:51.548585 2382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:51.565220 2382 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44547
I20260808 15:07:51.565277 2476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44547 every 8 connection(s)
I20260808 15:07:51.566493 2382 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:51.567253 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 2382
I20260808 15:07:51.567420 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:51.568041 2771 external_mini_cluster.cc:1428] Setting key 2c4c5c07630ef44750ff1aa8b9507c68
I20260808 15:07:51.570636 2478 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:51.575354 2481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:51.569773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:41621 (local address 127.0.0.1:44547)
0808 15:07:51.570523 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:51.570535 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:51.570574 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:07:51.572219 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:51.572241 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:51.572532 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:51.572876 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:51.573835 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:51.573844 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:51.573856 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:51.573876 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:51.573953 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:51.573972 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:51.574191 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:51.574325 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 15:07:51.574510 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":310,"threads_started":1}
I20260808 15:07:51.579258 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:51.582880 2478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733: Bootstrap starting.
I20260808 15:07:51.586562 2478 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:51.587872 2478 log.cc:824] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:51.590953 2478 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733: No bootstrap required, opened a new log
I20260808 15:07:51.591774 2481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:51.589451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.68:37667 (local address 127.0.0.1:44547)
0808 15:07:51.589619 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:51.589624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:51.589967 (+ 343us) server_negotiation.cc:407] Connection header received
0808 15:07:51.590129 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:51.590133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:51.590176 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:51.590250 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:51.591158 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:51.591162 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:51.591165 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:51.591180 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:51.591217 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:51.591231 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:51.591316 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:51.591453 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 15:07:51.591509 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:07:51.598726 2478 raft_consensus.cc:348] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825ca75139a94784a573c2100562c733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } }
I20260808 15:07:51.599031 2478 raft_consensus.cc:374] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:51.599134 2478 raft_consensus.cc:729] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 825ca75139a94784a573c2100562c733, State: Initialized, Role: FOLLOWER
I20260808 15:07:51.599640 2478 consensus_queue.cc:260] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825ca75139a94784a573c2100562c733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } }
I20260808 15:07:51.599862 2478 raft_consensus.cc:388] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:51.599996 2478 raft_consensus.cc:482] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:51.600158 2478 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:51.602531 2478 raft_consensus.cc:504] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825ca75139a94784a573c2100562c733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } }
I20260808 15:07:51.602977 2478 leader_election.cc:302] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 825ca75139a94784a573c2100562c733; no voters:
I20260808 15:07:51.603505 2478 leader_election.cc:288] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:51.603639 2488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:51.604247 2488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [term 1 LEADER]: Becoming Leader. State: Replica: 825ca75139a94784a573c2100562c733, State: Running, Role: LEADER
I20260808 15:07:51.604737 2488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825ca75139a94784a573c2100562c733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } }
I20260808 15:07:51.605346 2478 sys_catalog.cc:564] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:51.607415 2491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 825ca75139a94784a573c2100562c733. Latest consensus state: current_term: 1 leader_uuid: "825ca75139a94784a573c2100562c733" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825ca75139a94784a573c2100562c733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } } }
I20260808 15:07:51.607859 2491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:51.608480 2489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "825ca75139a94784a573c2100562c733" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "825ca75139a94784a573c2100562c733" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } } }
I20260808 15:07:51.608843 2489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:51.610075 2497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:51.615007 2497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:51.623282 2497 catalog_manager.cc:1347] Generated new cluster ID: 0b409266095c4e1588a3599f49c6ea54
I20260808 15:07:51.623385 2497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:51.642257 2497 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:51.642715 2497 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:51.651377 2497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 825ca75139a94784a573c2100562c733: Generated new TSK 0
I20260808 15:07:51.652297 2497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:51.662909 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2382
W20260808 15:07:51.663933 2466 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.68:41621
W20260808 15:07:51.664556 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2382: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:51.664712 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 2382
2026-08-08T15:07:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1947 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:51 dist-test-slave-jlzn krb5kdc[2525](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:51 dist-test-slave-jlzn krb5kdc[2525](info): set up 2 sockets
Aug 08 15:07:51 dist-test-slave-jlzn krb5kdc[2525](info): commencing operation
krb5kdc: starting...
W20260808 15:07:52.884198 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:53 dist-test-slave-jlzn krb5kdc[2525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:07:53.132602 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40367
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:42349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:53.447614 2539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:53.447921 2539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:53.448056 2539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:53.455073 2539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:53.455197 2539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:53.455279 2539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:53.455363 2539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:53.455422 2539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:53.465008 2539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:42349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40367
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:53.466210 2539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:53.467470 2539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:53.478534 2547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:53.478731 2546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:53.478502 2549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:53.479562 2539 server_base.cc:1034] running on GCE node
I20260808 15:07:53.480332 2539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:53.481627 2539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:53.482831 2539 hybrid_clock.cc:648] HybridClock initialized: now 1786201673482795 us; error 31 us; skew 500 ppm
I20260808 15:07:53.483119 2539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:53 dist-test-slave-jlzn krb5kdc[2525](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201673, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:53.486006 2539 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:07:53.487514 2539 webserver.cc:460] Webserver started at http://127.2.180.254:45657/ using document root <none> and password file <none>
I20260808 15:07:53.488168 2539 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:53.488293 2539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:53.488577 2539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:53.491393 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "b13c8c9a4b1e4773a6b7047645cef776"
format_stamp: "Formatted at 2026-08-08 15:07:53 on dist-test-slave-jlzn"
server_key: "1fea94c01b9a3d9f6826a5ee2135e1d5"
server_key_iv: "62103f9d96402019de752997e3f18ce3"
server_key_version: "encryptionkey@0"
I20260808 15:07:53.492216 2539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "b13c8c9a4b1e4773a6b7047645cef776"
format_stamp: "Formatted at 2026-08-08 15:07:53 on dist-test-slave-jlzn"
server_key: "1fea94c01b9a3d9f6826a5ee2135e1d5"
server_key_iv: "62103f9d96402019de752997e3f18ce3"
server_key_version: "encryptionkey@0"
I20260808 15:07:53.496889 2539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 15:07:53.500041 2564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:53.501209 2539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:07:53.501363 2539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "b13c8c9a4b1e4773a6b7047645cef776"
format_stamp: "Formatted at 2026-08-08 15:07:53 on dist-test-slave-jlzn"
server_key: "1fea94c01b9a3d9f6826a5ee2135e1d5"
server_key_iv: "62103f9d96402019de752997e3f18ce3"
server_key_version: "encryptionkey@0"
I20260808 15:07:53.501528 2539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:53.514071 2539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:53.517982 2539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:53.518239 2539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:53.535053 2539 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:40367
I20260808 15:07:53.535075 2632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:40367 every 8 connection(s)
I20260808 15:07:53.536331 2539 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:53.539731 2633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:53.542531 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 2539
I20260808 15:07:53.542682 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:53.543089 2771 external_mini_cluster.cc:1428] Setting key 35c0beea31b017b5420c8fc40b1fcbff
I20260808 15:07:53.550935 2633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776: Bootstrap starting.
I20260808 15:07:53.554553 2633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:53.555863 2633 log.cc:824] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776: Log is configured to *not* fsync() on all Append() calls
Aug 08 15:07:53 dist-test-slave-jlzn krb5kdc[2525](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:53.558914 2633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776: No bootstrap required, opened a new log
I20260808 15:07:53.565608 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:53.544779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48071 (local address 127.2.180.254:40367)
0808 15:07:53.545496 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:53.545506 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:53.545544 (+ 38us) server_negotiation.cc:407] Connection header received
0808 15:07:53.547139 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:53.547167 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:53.547520 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:53.547847 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:53.549580 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:53.550791 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:53.552553 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:53.553190 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:53.553309 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:53.558282 (+ 4973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:53.558289 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:53.558302 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:53.558339 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:53.561410 (+ 3071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:53.562441 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:53.562448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:53.562457 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:53.562669 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:53.563136 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:53.563140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:53.563142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:53.563665 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:53.563963 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:53.564375 (+ 412us) server_negotiation.cc:299] Negotiation successful
0808 15:07:53.564637 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":320,"threads_started":1}
I20260808 15:07:53.567711 2633 raft_consensus.cc:348] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13c8c9a4b1e4773a6b7047645cef776" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40367 } }
I20260808 15:07:53.568157 2633 raft_consensus.cc:374] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:53.568248 2633 raft_consensus.cc:729] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b13c8c9a4b1e4773a6b7047645cef776, State: Initialized, Role: FOLLOWER
I20260808 15:07:53.568797 2633 consensus_queue.cc:260] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13c8c9a4b1e4773a6b7047645cef776" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40367 } }
I20260808 15:07:53.569023 2633 raft_consensus.cc:388] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:53.569167 2633 raft_consensus.cc:482] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:53.569324 2633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:53.569376 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:53.572345 2633 raft_consensus.cc:504] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13c8c9a4b1e4773a6b7047645cef776" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40367 } }
I20260808 15:07:53.572929 2633 leader_election.cc:302] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b13c8c9a4b1e4773a6b7047645cef776; no voters:
I20260808 15:07:53.573573 2633 leader_election.cc:288] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:53.573650 2646 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:53.575686 2633 sys_catalog.cc:564] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:53.580286 2646 raft_consensus.cc:686] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [term 1 LEADER]: Becoming Leader. State: Replica: b13c8c9a4b1e4773a6b7047645cef776, State: Running, Role: LEADER
I20260808 15:07:53.580842 2646 consensus_queue.cc:237] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13c8c9a4b1e4773a6b7047645cef776" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40367 } }
I20260808 15:07:53.584173 2647 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b13c8c9a4b1e4773a6b7047645cef776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13c8c9a4b1e4773a6b7047645cef776" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40367 } } }
I20260808 15:07:53.584579 2647 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:53.585071 2648 sys_catalog.cc:455] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b13c8c9a4b1e4773a6b7047645cef776. Latest consensus state: current_term: 1 leader_uuid: "b13c8c9a4b1e4773a6b7047645cef776" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b13c8c9a4b1e4773a6b7047645cef776" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40367 } } }
I20260808 15:07:53.585355 2648 sys_catalog.cc:458] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:53.586475 2665 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:53.590667 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:53.574884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39981 (local address 127.2.180.254:40367)
0808 15:07:53.575046 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:53.575052 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:53.575285 (+ 233us) server_negotiation.cc:407] Connection header received
0808 15:07:53.575448 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:53.575453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:53.575523 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:53.575626 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:53.577544 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:53.578809 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:53.580585 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:53.581286 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:53.581420 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:53.584463 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:53.584470 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:53.584474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:53.584510 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:53.588009 (+ 3499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:53.588669 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:53.588673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:53.588675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:53.588794 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:53.589242 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:53.589245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:53.589247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:53.589576 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:53.589849 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:53.590226 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 15:07:53.590400 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 15:07:53.592317 2665 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:53.600479 2665 catalog_manager.cc:1347] Generated new cluster ID: 351dd435b3734d2f8cfb819c0e44abca
I20260808 15:07:53.600574 2665 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:53.620846 2665 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:53.622071 2665 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:53.631057 2665 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b13c8c9a4b1e4773a6b7047645cef776: Generated new TSK 0
I20260808 15:07:53.631965 2665 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:53.646409 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2539
I20260808 15:07:53.647459 2628 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:53.742323 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 2539
W20260808 15:07:53.767263 2523 connection.cc:535] client connection to 127.2.180.254:40367 recv error: Network error: failed to read from TLS socket (remote: 127.2.180.254:40367): Connection reset by peer (error 104)
2026-08-08T15:07:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2078 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:53 dist-test-slave-jlzn krb5kdc[2673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:53 dist-test-slave-jlzn krb5kdc[2673](info): set up 2 sockets
Aug 08 15:07:53 dist-test-slave-jlzn krb5kdc[2673](info): commencing operation
krb5kdc: starting...
W20260808 15:07:54.934960 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.002s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:55 dist-test-slave-jlzn krb5kdc[2673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:07:55.190749 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:44245
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:43215
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:44245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:55.510340 2693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:55.510603 2693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:55.510700 2693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:55.517628 2693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:55.517730 2693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:55.517781 2693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:55.517840 2693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:55.517884 2693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:55.527107 2693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:43215
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:44245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:44245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:55.528211 2693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:55.529403 2693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:55.540496 2698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:55.540518 2699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:55.540518 2702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:55.541499 2693 server_base.cc:1034] running on GCE node
I20260808 15:07:55.542259 2693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:55.543519 2693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:55.544710 2693 hybrid_clock.cc:648] HybridClock initialized: now 1786201675544666 us; error 37 us; skew 500 ppm
I20260808 15:07:55.545023 2693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:55 dist-test-slave-jlzn krb5kdc[2673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201675, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:55.548143 2693 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:07:55.549429 2693 webserver.cc:460] Webserver started at http://127.2.180.254:40941/ using document root <none> and password file <none>
I20260808 15:07:55.550021 2693 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:55.550127 2693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:55.550370 2693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:55.553244 2693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426"
format_stamp: "Formatted at 2026-08-08 15:07:55 on dist-test-slave-jlzn"
server_key: "84de439ec54388566da6cbbd32bc25b5"
server_key_iv: "b884397a919d67d79047dcedd3de8c1d"
server_key_version: "encryptionkey@0"
I20260808 15:07:55.554028 2693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426"
format_stamp: "Formatted at 2026-08-08 15:07:55 on dist-test-slave-jlzn"
server_key: "84de439ec54388566da6cbbd32bc25b5"
server_key_iv: "b884397a919d67d79047dcedd3de8c1d"
server_key_version: "encryptionkey@0"
I20260808 15:07:55.558838 2693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 15:07:55.562156 2712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:55.563380 2693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 15:07:55.563519 2693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426"
format_stamp: "Formatted at 2026-08-08 15:07:55 on dist-test-slave-jlzn"
server_key: "84de439ec54388566da6cbbd32bc25b5"
server_key_iv: "b884397a919d67d79047dcedd3de8c1d"
server_key_version: "encryptionkey@0"
I20260808 15:07:55.563699 2693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:55.580045 2693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:55.583582 2693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:55.583796 2693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:55.600708 2693 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:44245
I20260808 15:07:55.600740 2793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:44245 every 8 connection(s)
I20260808 15:07:55.601989 2693 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:55.605549 2794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:55.611738 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 2693
I20260808 15:07:55.611923 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:55.612354 2771 external_mini_cluster.cc:1428] Setting key aef469b4ef69a27c478ce19718960f9f
I20260808 15:07:55.616078 2794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426: Bootstrap starting.
I20260808 15:07:55.619230 2794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:55.620456 2794 log.cc:824] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:55.623548 2794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426: No bootstrap required, opened a new log
I20260808 15:07:55.625619 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:55.614091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52099 (local address 127.2.180.254:44245)
0808 15:07:55.614842 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:55.614852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:55.614892 (+ 40us) server_negotiation.cc:407] Connection header received
0808 15:07:55.616630 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:55.616653 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:55.616997 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:55.617328 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:55.617900 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:55.619186 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:55.621012 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:55.621984 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:55.622119 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:55.623346 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:55.623354 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:55.623367 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:55.623971 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:55.624047 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:55.624083 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:55.624215 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:55.624481 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 15:07:55.624680 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":341,"threads_started":1}
I20260808 15:07:55.629292 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:55.631556 2794 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44245 } }
I20260808 15:07:55.631943 2794 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:55.632046 2794 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0f78fd5a25f4e5c81c4ccf1dc2cd426, State: Initialized, Role: FOLLOWER
I20260808 15:07:55.632586 2794 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44245 } }
I20260808 15:07:55.632802 2794 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:55.632925 2794 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:55.633087 2794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:55.635237 2794 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44245 } }
I20260808 15:07:55.635689 2794 leader_election.cc:302] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0f78fd5a25f4e5c81c4ccf1dc2cd426; no voters:
I20260808 15:07:55.636268 2794 leader_election.cc:288] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:55.636365 2811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:55.638106 2811 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [term 1 LEADER]: Becoming Leader. State: Replica: e0f78fd5a25f4e5c81c4ccf1dc2cd426, State: Running, Role: LEADER
I20260808 15:07:55.638687 2811 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44245 } }
I20260808 15:07:55.638975 2794 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:55.641203 2814 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0f78fd5a25f4e5c81c4ccf1dc2cd426. Latest consensus state: current_term: 1 leader_uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44245 } } }
I20260808 15:07:55.641569 2814 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:55.642177 2812 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0f78fd5a25f4e5c81c4ccf1dc2cd426" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 44245 } } }
I20260808 15:07:55.642479 2812 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:55.643036 2818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:55.643107 2798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:55.633450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47511 (local address 127.2.180.254:44245)
0808 15:07:55.633653 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:55.633659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:55.636319 (+ 2660us) server_negotiation.cc:407] Connection header received
0808 15:07:55.636532 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:55.636536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:55.636597 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:55.636690 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:55.637221 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:55.638357 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:55.639753 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:55.640437 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:55.640562 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:55.641841 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:55.641849 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:55.641853 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:55.641878 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:55.641923 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:55.641945 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:55.642108 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:55.642725 (+ 617us) server_negotiation.cc:299] Negotiation successful
0808 15:07:55.642788 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 15:07:55.648255 2818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:55.659581 2818 catalog_manager.cc:1347] Generated new cluster ID: 3d045e7e4bcd428396986d24a1f1b480
I20260808 15:07:55.659700 2818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:55.680331 2818 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:55.681443 2818 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:55.694281 2818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0f78fd5a25f4e5c81c4ccf1dc2cd426: Generated new TSK 0
I20260808 15:07:55.694949 2818 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:55.710451 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2693
W20260808 15:07:55.711661 2787 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52099
W20260808 15:07:55.712330 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2693: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:55.712458 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 2693
W20260808 15:07:55.737413 2671 connection.cc:535] client connection to 127.2.180.254:44245 recv error: Network error: failed to read from TLS socket (remote: 127.2.180.254:44245): Connection reset by peer (error 104)
2026-08-08T15:07:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1970 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:55 dist-test-slave-jlzn krb5kdc[2840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:55 dist-test-slave-jlzn krb5kdc[2840](info): set up 2 sockets
Aug 08 15:07:55 dist-test-slave-jlzn krb5kdc[2840](info): commencing operation
krb5kdc: starting...
W20260808 15:07:56.912236 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.006s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:57 dist-test-slave-jlzn krb5kdc[2840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:07:57.175325 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35011
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:41963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:35011
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:57.496690 2856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:57.496946 2856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:57.497042 2856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:57.503839 2856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:57.503945 2856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:57.503998 2856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:57.504057 2856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:57.504099 2856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:57.513325 2856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:41963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:35011
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:35011
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:57.514362 2856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:57.515538 2856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:57.526501 2867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:57.526532 2864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:57.526500 2865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:57.527320 2856 server_base.cc:1034] running on GCE node
I20260808 15:07:57.528098 2856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:57.529426 2856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:57.530628 2856 hybrid_clock.cc:648] HybridClock initialized: now 1786201677530583 us; error 33 us; skew 500 ppm
I20260808 15:07:57.530934 2856 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:57 dist-test-slave-jlzn krb5kdc[2840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201677, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:57.533903 2856 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:07:57.535264 2856 webserver.cc:460] Webserver started at http://127.2.180.254:37903/ using document root <none> and password file <none>
I20260808 15:07:57.535907 2856 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:57.536036 2856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:57.536330 2856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:57.539212 2856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "8114252fb9884bd7a82df90dc0ea6ea2"
format_stamp: "Formatted at 2026-08-08 15:07:57 on dist-test-slave-jlzn"
server_key: "f80606a82810288458216186573a002b"
server_key_iv: "ace5ab05d290f4115a8e692a016a9bd4"
server_key_version: "encryptionkey@0"
I20260808 15:07:57.539996 2856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "8114252fb9884bd7a82df90dc0ea6ea2"
format_stamp: "Formatted at 2026-08-08 15:07:57 on dist-test-slave-jlzn"
server_key: "f80606a82810288458216186573a002b"
server_key_iv: "ace5ab05d290f4115a8e692a016a9bd4"
server_key_version: "encryptionkey@0"
I20260808 15:07:57.544935 2856 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 15:07:57.548161 2876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:57.549350 2856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260808 15:07:57.549502 2856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "8114252fb9884bd7a82df90dc0ea6ea2"
format_stamp: "Formatted at 2026-08-08 15:07:57 on dist-test-slave-jlzn"
server_key: "f80606a82810288458216186573a002b"
server_key_iv: "ace5ab05d290f4115a8e692a016a9bd4"
server_key_version: "encryptionkey@0"
I20260808 15:07:57.549676 2856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:57.554998 2856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:57.558557 2856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:57.558820 2856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:57.575037 2856 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:35011
I20260808 15:07:57.575079 2949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:35011 every 8 connection(s)
I20260808 15:07:57.576292 2856 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:57.579782 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 2856
I20260808 15:07:57.579865 2951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:57.579986 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:57.580511 2771 external_mini_cluster.cc:1428] Setting key d22c2c82023a02ae720b4bac7d102a01
I20260808 15:07:57.587637 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:57.582179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58579 (local address 127.2.180.254:35011)
0808 15:07:57.582985 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:57.582996 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:57.583040 (+ 44us) server_negotiation.cc:407] Connection header received
0808 15:07:57.584582 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:57.584605 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:57.584918 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:57.585231 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:57.586180 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:57.586185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:57.586198 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:57.586218 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:57.586272 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:57.586293 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:57.586459 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:57.586561 (+ 102us) server_negotiation.cc:299] Negotiation successful
0808 15:07:57.586728 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":371,"threads_started":1}
I20260808 15:07:57.591212 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:57.593191 2951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2: Bootstrap starting.
I20260808 15:07:57.596684 2951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:57.597765 2951 log.cc:824] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:57.598984 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:57.596176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33317 (local address 127.2.180.254:35011)
0808 15:07:57.596332 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:57.596337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:57.597288 (+ 951us) server_negotiation.cc:407] Connection header received
0808 15:07:57.597395 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:57.597400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:57.597458 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:57.597558 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:57.598386 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:57.598390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:57.598393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:07:57.598407 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:57.598440 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:07:57.598453 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:57.598541 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:57.598704 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 15:07:57.598746 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:07:57.600615 2951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2: No bootstrap required, opened a new log
I20260808 15:07:57.607589 2951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114252fb9884bd7a82df90dc0ea6ea2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35011 } }
I20260808 15:07:57.607928 2951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:57.608022 2951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8114252fb9884bd7a82df90dc0ea6ea2, State: Initialized, Role: FOLLOWER
I20260808 15:07:57.608525 2951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114252fb9884bd7a82df90dc0ea6ea2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35011 } }
I20260808 15:07:57.608748 2951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:57.608884 2951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:57.609030 2951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:57.611002 2951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114252fb9884bd7a82df90dc0ea6ea2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35011 } }
I20260808 15:07:57.611446 2951 leader_election.cc:302] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8114252fb9884bd7a82df90dc0ea6ea2; no voters:
I20260808 15:07:57.612048 2951 leader_election.cc:288] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:57.612154 2968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:57.613262 2968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [term 1 LEADER]: Becoming Leader. State: Replica: 8114252fb9884bd7a82df90dc0ea6ea2, State: Running, Role: LEADER
I20260808 15:07:57.613767 2968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114252fb9884bd7a82df90dc0ea6ea2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35011 } }
I20260808 15:07:57.614050 2951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:57.616370 2973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8114252fb9884bd7a82df90dc0ea6ea2. Latest consensus state: current_term: 1 leader_uuid: "8114252fb9884bd7a82df90dc0ea6ea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114252fb9884bd7a82df90dc0ea6ea2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35011 } } }
I20260808 15:07:57.616418 2970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8114252fb9884bd7a82df90dc0ea6ea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8114252fb9884bd7a82df90dc0ea6ea2" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 35011 } } }
I20260808 15:07:57.616851 2973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:57.616860 2970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:57.617753 2985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:57.622751 2985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:57.633512 2985 catalog_manager.cc:1347] Generated new cluster ID: 9124cad4de7a4328a9fcd002510a13e8
I20260808 15:07:57.633638 2985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:57.644089 2985 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:57.644531 2985 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:57.656278 2985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8114252fb9884bd7a82df90dc0ea6ea2: Generated new TSK 0
I20260808 15:07:57.657156 2985 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:57.661118 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2856
W20260808 15:07:57.662153 2943 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58579
W20260808 15:07:57.662775 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 2856: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:07:57.662925 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 2856
2026-08-08T15:07:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1947 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:57 dist-test-slave-jlzn krb5kdc[3003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:57 dist-test-slave-jlzn krb5kdc[3003](info): set up 2 sockets
Aug 08 15:07:57 dist-test-slave-jlzn krb5kdc[3003](info): commencing operation
krb5kdc: starting...
W20260808 15:07:58.845247 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:07:59 dist-test-slave-jlzn krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:07:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:07:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:07:59.095088 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:39685
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:34753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:39685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:07:59.413223 3019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:07:59.413486 3019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:07:59.413590 3019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:07:59.420661 3019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:07:59.420764 3019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:07:59.420818 3019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:07:59.420877 3019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:07:59.420920 3019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:07:59.430284 3019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:34753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:39685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:39685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:07:59.431354 3019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:07:59.432605 3019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:07:59.443253 3028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:59.443286 3029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:07:59.443480 3032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:07:59.444432 3019 server_base.cc:1034] running on GCE node
I20260808 15:07:59.445302 3019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:07:59.446579 3019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:07:59.447779 3019 hybrid_clock.cc:648] HybridClock initialized: now 1786201679447737 us; error 29 us; skew 500 ppm
I20260808 15:07:59.448128 3019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:07:59 dist-test-slave-jlzn krb5kdc[3003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201679, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:07:59.453778 3019 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:07:59.455169 3019 webserver.cc:460] Webserver started at http://127.2.180.254:43217/ using document root <none> and password file <none>
I20260808 15:07:59.455778 3019 fs_manager.cc:359] Metadata directory not provided
I20260808 15:07:59.455941 3019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:07:59.456224 3019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:07:59.459105 3019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "37800789d25e41098eb278431b2cc861"
format_stamp: "Formatted at 2026-08-08 15:07:59 on dist-test-slave-jlzn"
server_key: "c7fac17347d06ab036df0c8588b9293e"
server_key_iv: "012007b91268749d463e1cdc09244721"
server_key_version: "encryptionkey@0"
I20260808 15:07:59.459867 3019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "37800789d25e41098eb278431b2cc861"
format_stamp: "Formatted at 2026-08-08 15:07:59 on dist-test-slave-jlzn"
server_key: "c7fac17347d06ab036df0c8588b9293e"
server_key_iv: "012007b91268749d463e1cdc09244721"
server_key_version: "encryptionkey@0"
I20260808 15:07:59.464584 3019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260808 15:07:59.467819 3043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:07:59.469012 3019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 15:07:59.469161 3019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "37800789d25e41098eb278431b2cc861"
format_stamp: "Formatted at 2026-08-08 15:07:59 on dist-test-slave-jlzn"
server_key: "c7fac17347d06ab036df0c8588b9293e"
server_key_iv: "012007b91268749d463e1cdc09244721"
server_key_version: "encryptionkey@0"
I20260808 15:07:59.469326 3019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:07:59.488628 3019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:07:59.492695 3019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:07:59.492936 3019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:07:59.509940 3019 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:39685
I20260808 15:07:59.509948 3119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:39685 every 8 connection(s)
I20260808 15:07:59.511175 3019 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:07:59.514628 3125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:07:59.515234 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3019
I20260808 15:07:59.515410 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:07:59.516001 2771 external_mini_cluster.cc:1428] Setting key edd0eb596dfa409a1cf526afa2930314
I20260808 15:07:59.527674 3125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861: Bootstrap starting.
Aug 08 15:07:59 dist-test-slave-jlzn krb5kdc[3003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:07:59.531024 3125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861: Neither blocks nor log segments found. Creating new log.
I20260808 15:07:59.532384 3125 log.cc:824] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861: Log is configured to *not* fsync() on all Append() calls
I20260808 15:07:59.535213 3125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861: No bootstrap required, opened a new log
I20260808 15:07:59.538142 3129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:59.517706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35567 (local address 127.2.180.254:39685)
0808 15:07:59.518446 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:59.518456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:59.518497 (+ 41us) server_negotiation.cc:407] Connection header received
0808 15:07:59.520525 (+ 2028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:59.520546 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:59.520911 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:59.521224 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:59.522861 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:59.524159 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:59.525927 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:59.526552 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:59.526688 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:59.531108 (+ 4420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:59.531119 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:59.531137 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:59.531179 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:59.534322 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:59.535103 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:59.535109 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:59.535115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:59.535276 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:59.535670 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:59.535673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:59.535675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:59.536187 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:59.536403 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:59.536840 (+ 437us) server_negotiation.cc:299] Negotiation successful
0808 15:07:59.537157 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":314,"threads_started":1}
I20260808 15:07:59.541487 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:07:59.543640 3125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37800789d25e41098eb278431b2cc861" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39685 } }
I20260808 15:07:59.544075 3125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:07:59.544188 3125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37800789d25e41098eb278431b2cc861, State: Initialized, Role: FOLLOWER
I20260808 15:07:59.544803 3125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37800789d25e41098eb278431b2cc861" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39685 } }
I20260808 15:07:59.545092 3125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:07:59.545250 3125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:07:59.545432 3125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:07:59.548226 3125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37800789d25e41098eb278431b2cc861" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39685 } }
I20260808 15:07:59.548787 3125 leader_election.cc:302] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37800789d25e41098eb278431b2cc861; no voters:
I20260808 15:07:59.549425 3125 leader_election.cc:288] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:07:59.549546 3138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:07:59.550143 3138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [term 1 LEADER]: Becoming Leader. State: Replica: 37800789d25e41098eb278431b2cc861, State: Running, Role: LEADER
I20260808 15:07:59.550724 3138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37800789d25e41098eb278431b2cc861" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39685 } }
I20260808 15:07:59.553289 3125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:07:59.554620 3140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37800789d25e41098eb278431b2cc861. Latest consensus state: current_term: 1 leader_uuid: "37800789d25e41098eb278431b2cc861" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37800789d25e41098eb278431b2cc861" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39685 } } }
I20260808 15:07:59.554610 3141 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37800789d25e41098eb278431b2cc861" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37800789d25e41098eb278431b2cc861" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 39685 } } }
I20260808 15:07:59.555086 3141 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:59.555087 3140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861 [sys.catalog]: This master's current role is: LEADER
I20260808 15:07:59.555923 3150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:07:59.563320 3129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:07:59.546528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54661 (local address 127.2.180.254:39685)
0808 15:07:59.546717 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 15:07:59.546723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:07:59.547202 (+ 479us) server_negotiation.cc:407] Connection header received
0808 15:07:59.547348 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:07:59.547352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:07:59.547406 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:07:59.547477 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:07:59.549214 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:59.550331 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:59.552095 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:07:59.552795 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:07:59.552921 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:07:59.555783 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:07:59.555789 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:07:59.555795 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:07:59.555855 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:07:59.559523 (+ 3668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:59.560387 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:59.560393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:59.560397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:59.560543 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:07:59.560982 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:07:59.560988 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:07:59.560992 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:07:59.561411 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:07:59.561667 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 15:07:59.562745 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0808 15:07:59.562964 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:07:59.573796 3150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:07:59.583469 3150 catalog_manager.cc:1347] Generated new cluster ID: c0717b4bbf1343d9bc5494597bce64ec
I20260808 15:07:59.583577 3150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:07:59.594619 3150 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:07:59.595831 3150 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:07:59.607861 3150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37800789d25e41098eb278431b2cc861: Generated new TSK 0
I20260808 15:07:59.608628 3150 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:07:59.618943 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3019
I20260808 15:07:59.620033 3113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:07:59.708371 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3019
2026-08-08T15:07:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2043 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:07:59 dist-test-slave-jlzn krb5kdc[3175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:07:59 dist-test-slave-jlzn krb5kdc[3175](info): set up 2 sockets
Aug 08 15:07:59 dist-test-slave-jlzn krb5kdc[3175](info): commencing operation
krb5kdc: starting...
W20260808 15:08:00.896354 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.010s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:08:01 dist-test-slave-jlzn krb5kdc[3175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:08:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:08:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:08:01.140448 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:38055
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:38971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:38055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:08:01.458791 3190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:08:01.459066 3190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:08:01.459210 3190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:08:01.466071 3190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:08:01.466177 3190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:08:01.466254 3190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:08:01.466333 3190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:08:01.466396 3190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:08:01.475562 3190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:38971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:38055
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:38055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:08:01.476703 3190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:08:01.477921 3190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:08:01.489262 3197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:01.489219 3199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:01.489296 3202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:08:01.490041 3190 server_base.cc:1034] running on GCE node
I20260808 15:08:01.490744 3190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:08:01.492038 3190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:08:01.493232 3190 hybrid_clock.cc:648] HybridClock initialized: now 1786201681493188 us; error 38 us; skew 500 ppm
I20260808 15:08:01.493535 3190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:08:01 dist-test-slave-jlzn krb5kdc[3175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201681, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:08:01.496634 3190 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:08:01.497900 3190 webserver.cc:460] Webserver started at http://127.2.180.254:42395/ using document root <none> and password file <none>
I20260808 15:08:01.498484 3190 fs_manager.cc:359] Metadata directory not provided
I20260808 15:08:01.498587 3190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:08:01.498826 3190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:08:01.501621 3190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "fcc94f704b5243ef854cd5c4a53a36f4"
format_stamp: "Formatted at 2026-08-08 15:08:01 on dist-test-slave-jlzn"
server_key: "f42c481040ef2b3592b2b59e23da748c"
server_key_iv: "c93318a2ad7eb9840d9cb554cdb6547d"
server_key_version: "encryptionkey@0"
I20260808 15:08:01.502326 3190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "fcc94f704b5243ef854cd5c4a53a36f4"
format_stamp: "Formatted at 2026-08-08 15:08:01 on dist-test-slave-jlzn"
server_key: "f42c481040ef2b3592b2b59e23da748c"
server_key_iv: "c93318a2ad7eb9840d9cb554cdb6547d"
server_key_version: "encryptionkey@0"
I20260808 15:08:01.507272 3190 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 15:08:01.510502 3211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:08:01.511684 3190 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:08:01.511842 3190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "fcc94f704b5243ef854cd5c4a53a36f4"
format_stamp: "Formatted at 2026-08-08 15:08:01 on dist-test-slave-jlzn"
server_key: "f42c481040ef2b3592b2b59e23da748c"
server_key_iv: "c93318a2ad7eb9840d9cb554cdb6547d"
server_key_version: "encryptionkey@0"
I20260808 15:08:01.512032 3190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:08:01.524892 3190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:08:01.528669 3190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:08:01.528888 3190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:08:01.545192 3190 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:38055
I20260808 15:08:01.545220 3290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:38055 every 8 connection(s)
I20260808 15:08:01.546461 3190 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:08:01.549846 3291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:08:01.550874 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3190
I20260808 15:08:01.551064 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:08:01.551628 2771 external_mini_cluster.cc:1428] Setting key de06623a6ac5011fb8989fb409f05ea6
I20260808 15:08:01.562696 3291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4: Bootstrap starting.
I20260808 15:08:01.564095 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:01.553322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50823 (local address 127.2.180.254:38055)
0808 15:08:01.554045 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:01.554059 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:01.554098 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:08:01.555763 (+ 1665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:01.555787 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:01.556198 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:01.556495 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:01.557078 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:01.558269 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:01.560077 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:01.560737 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:01.560866 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:01.561995 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:01.562002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:01.562014 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:08:01.562492 (+ 478us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:01.562567 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:08:01.562592 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:01.562671 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:01.562919 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 15:08:01.563150 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":325,"threads_started":1}
I20260808 15:08:01.565902 3291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4: Neither blocks nor log segments found. Creating new log.
I20260808 15:08:01.567076 3291 log.cc:824] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4: Log is configured to *not* fsync() on all Append() calls
I20260808 15:08:01.567559 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:08:01.569932 3291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4: No bootstrap required, opened a new log
I20260808 15:08:01.578313 3295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:01.572235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50907 (local address 127.2.180.254:38055)
0808 15:08:01.572397 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:01.572402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:01.572845 (+ 443us) server_negotiation.cc:407] Connection header received
0808 15:08:01.572973 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:01.572978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:01.573047 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:01.573161 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:01.573588 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:01.574656 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:01.575967 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:01.576664 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:01.576780 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:01.577651 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:01.577658 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:01.577662 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:08:01.577685 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:01.577734 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:08:01.577755 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:01.577841 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:01.577923 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 15:08:01.578012 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 15:08:01.579291 3291 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc94f704b5243ef854cd5c4a53a36f4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 38055 } }
I20260808 15:08:01.579702 3291 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:08:01.579797 3291 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc94f704b5243ef854cd5c4a53a36f4, State: Initialized, Role: FOLLOWER
I20260808 15:08:01.580345 3291 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc94f704b5243ef854cd5c4a53a36f4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 38055 } }
I20260808 15:08:01.580570 3291 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:08:01.580703 3291 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:08:01.580844 3291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:08:01.582887 3291 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc94f704b5243ef854cd5c4a53a36f4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 38055 } }
I20260808 15:08:01.583359 3291 leader_election.cc:302] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcc94f704b5243ef854cd5c4a53a36f4; no voters:
I20260808 15:08:01.583993 3291 leader_election.cc:288] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:08:01.584074 3304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:08:01.584728 3304 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [term 1 LEADER]: Becoming Leader. State: Replica: fcc94f704b5243ef854cd5c4a53a36f4, State: Running, Role: LEADER
I20260808 15:08:01.585234 3304 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc94f704b5243ef854cd5c4a53a36f4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 38055 } }
I20260808 15:08:01.585927 3291 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:08:01.588032 3305 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcc94f704b5243ef854cd5c4a53a36f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc94f704b5243ef854cd5c4a53a36f4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 38055 } } }
I20260808 15:08:01.588439 3305 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:01.590915 3313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:08:01.590816 3306 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcc94f704b5243ef854cd5c4a53a36f4. Latest consensus state: current_term: 1 leader_uuid: "fcc94f704b5243ef854cd5c4a53a36f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc94f704b5243ef854cd5c4a53a36f4" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 38055 } } }
I20260808 15:08:01.591143 3306 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4 [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:01.595439 3313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:08:01.607630 3313 catalog_manager.cc:1347] Generated new cluster ID: d44e50a07a78418e84867707859e5ea8
I20260808 15:08:01.607735 3313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:08:01.621174 3313 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:08:01.622850 3313 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:08:01.633853 3313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcc94f704b5243ef854cd5c4a53a36f4: Generated new TSK 0
I20260808 15:08:01.634626 3313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:08:01.646617 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3190
W20260808 15:08:01.647616 3285 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50823
W20260808 15:08:01.648305 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3190: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:08:01.648438 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3190
2026-08-08T15:08:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1941 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:08:01 dist-test-slave-jlzn krb5kdc[3331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:08:01 dist-test-slave-jlzn krb5kdc[3331](info): set up 2 sockets
Aug 08 15:08:01 dist-test-slave-jlzn krb5kdc[3331](info): commencing operation
krb5kdc: starting...
W20260808 15:08:02.830567 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:08:03 dist-test-slave-jlzn krb5kdc[3331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:08:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:08:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:08:03.108920 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:33805
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:33269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:33805
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:08:03.428634 3349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:08:03.428920 3349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:08:03.429054 3349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:08:03.436102 3349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:08:03.436215 3349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:08:03.436291 3349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:08:03.436370 3349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:08:03.436427 3349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:08:03.445851 3349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:33269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:33805
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:33805
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:08:03.447031 3349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:08:03.448304 3349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:08:03.459168 3354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:03.459208 3355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:03.459406 3358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:08:03.460311 3349 server_base.cc:1034] running on GCE node
I20260808 15:08:03.461019 3349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:08:03.462312 3349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:08:03.463501 3349 hybrid_clock.cc:648] HybridClock initialized: now 1786201683463451 us; error 40 us; skew 500 ppm
I20260808 15:08:03.463850 3349 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:08:03 dist-test-slave-jlzn krb5kdc[3331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201683, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:08:03.466737 3349 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:08:03.468189 3349 webserver.cc:460] Webserver started at http://127.2.180.254:42963/ using document root <none> and password file <none>
I20260808 15:08:03.468818 3349 fs_manager.cc:359] Metadata directory not provided
I20260808 15:08:03.468938 3349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:08:03.469237 3349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:08:03.472138 3349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "3d23e26001784d789fe1cf6815236ecd"
format_stamp: "Formatted at 2026-08-08 15:08:03 on dist-test-slave-jlzn"
server_key: "02134fc022f14b11619ddc4e7c65435f"
server_key_iv: "c090f60157497f58953997d42d5adb2f"
server_key_version: "encryptionkey@0"
I20260808 15:08:03.472887 3349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "3d23e26001784d789fe1cf6815236ecd"
format_stamp: "Formatted at 2026-08-08 15:08:03 on dist-test-slave-jlzn"
server_key: "02134fc022f14b11619ddc4e7c65435f"
server_key_iv: "c090f60157497f58953997d42d5adb2f"
server_key_version: "encryptionkey@0"
I20260808 15:08:03.477701 3349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 15:08:03.480968 3368 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:08:03.482159 3349 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 15:08:03.482321 3349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "3d23e26001784d789fe1cf6815236ecd"
format_stamp: "Formatted at 2026-08-08 15:08:03 on dist-test-slave-jlzn"
server_key: "02134fc022f14b11619ddc4e7c65435f"
server_key_iv: "c090f60157497f58953997d42d5adb2f"
server_key_version: "encryptionkey@0"
I20260808 15:08:03.482493 3349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:08:03.488184 3349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:08:03.492089 3349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:08:03.492353 3349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:08:03.509647 3349 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:33805
I20260808 15:08:03.509676 3444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:33805 every 8 connection(s)
I20260808 15:08:03.510933 3349 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:08:03.514552 3445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:08:03.521287 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3349
I20260808 15:08:03.521473 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:08:03.522030 2771 external_mini_cluster.cc:1428] Setting key 283965ea08db613b4bb7f664564f6975
I20260808 15:08:03.525453 3445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd: Bootstrap starting.
I20260808 15:08:03.528851 3445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd: Neither blocks nor log segments found. Creating new log.
I20260808 15:08:03.529567 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:03.523760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53027 (local address 127.2.180.254:33805)
0808 15:08:03.524574 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:03.524586 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:03.524625 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:08:03.526378 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:03.526400 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:03.526709 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:03.527024 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:03.528074 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:03.528080 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:03.528092 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:08:03.528108 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:03.528169 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:08:03.528188 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:03.528323 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:03.528465 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 15:08:03.528628 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":330,"threads_started":1}
I20260808 15:08:03.529907 3445 log.cc:824] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd: Log is configured to *not* fsync() on all Append() calls
I20260808 15:08:03.532661 3445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd: No bootstrap required, opened a new log
I20260808 15:08:03.532979 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:08:03.540110 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:03.537777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58283 (local address 127.2.180.254:33805)
0808 15:08:03.537947 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:03.537952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:03.538409 (+ 457us) server_negotiation.cc:407] Connection header received
0808 15:08:03.538533 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:03.538539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:03.538599 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:03.538701 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:03.539479 (+ 778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:03.539485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:03.539490 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 15:08:03.539506 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:03.539549 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 15:08:03.539572 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:03.539697 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:03.539743 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 15:08:03.539793 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 15:08:03.541985 3445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d23e26001784d789fe1cf6815236ecd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33805 } }
I20260808 15:08:03.542299 3445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 15:08:03.542369 3445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d23e26001784d789fe1cf6815236ecd, State: Initialized, Role: FOLLOWER
I20260808 15:08:03.542810 3445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d23e26001784d789fe1cf6815236ecd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33805 } }
I20260808 15:08:03.543013 3445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 15:08:03.543108 3445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 15:08:03.543231 3445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:08:03.545506 3445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d23e26001784d789fe1cf6815236ecd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33805 } }
I20260808 15:08:03.545984 3445 leader_election.cc:302] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d23e26001784d789fe1cf6815236ecd; no voters:
I20260808 15:08:03.546447 3445 leader_election.cc:288] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 15:08:03.546581 3460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:08:03.547186 3460 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [term 1 LEADER]: Becoming Leader. State: Replica: 3d23e26001784d789fe1cf6815236ecd, State: Running, Role: LEADER
I20260808 15:08:03.547669 3460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d23e26001784d789fe1cf6815236ecd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33805 } }
I20260808 15:08:03.548398 3445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:08:03.552091 3463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d23e26001784d789fe1cf6815236ecd. Latest consensus state: current_term: 1 leader_uuid: "3d23e26001784d789fe1cf6815236ecd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d23e26001784d789fe1cf6815236ecd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33805 } } }
I20260808 15:08:03.552420 3463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:03.558188 3462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d23e26001784d789fe1cf6815236ecd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d23e26001784d789fe1cf6815236ecd" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33805 } } }
I20260808 15:08:03.558485 3462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:03.564939 3476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:08:03.568924 3476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:08:03.576192 3476 catalog_manager.cc:1347] Generated new cluster ID: 2a05bfba2a5c418f90f89581c409ed1e
I20260808 15:08:03.576296 3476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:08:03.588176 3476 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:08:03.588627 3476 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:08:03.597831 3476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d23e26001784d789fe1cf6815236ecd: Generated new TSK 0
I20260808 15:08:03.598686 3476 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:08:03.612586 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3349
W20260808 15:08:03.613740 3438 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53027
W20260808 15:08:03.614367 2771 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3349: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 15:08:03.614523 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3349
2026-08-08T15:08:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1975 ms)
[----------] 18 tests from AuthTokenIssuingTest (35783 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:08:03 dist-test-slave-jlzn krb5kdc[3497](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:08:03 dist-test-slave-jlzn krb5kdc[3497](info): set up 2 sockets
Aug 08 15:08:03 dist-test-slave-jlzn krb5kdc[3497](info): commencing operation
krb5kdc: starting...
W20260808 15:08:04.818377 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:08:04 dist-test-slave-jlzn krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:08:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:08:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:08:05.088378 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40683
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:37791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40683,127.2.180.253:42213,127.2.180.252:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:08:05.406992 3516 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:08:05.407287 3516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:08:05.407383 3516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:08:05.407491 3516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:08:05.414482 3516 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:08:05.414575 3516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:08:05.414661 3516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:08:05.414718 3516 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:08:05.414803 3516 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:08:05.424229 3516 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:37791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40683,127.2.180.253:42213,127.2.180.252:44873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:40683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:08:05.425422 3516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:08:05.426676 3516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:08:05.437876 3521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:05.437935 3524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:05.437878 3529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:08:05.438679 3516 server_base.cc:1034] running on GCE node
I20260808 15:08:05.439378 3516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:08:05.440670 3516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:08:05.441902 3516 hybrid_clock.cc:648] HybridClock initialized: now 1786201685441830 us; error 42 us; skew 500 ppm
I20260808 15:08:05.442207 3516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:08:05 dist-test-slave-jlzn krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:08:05.445256 3516 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:08:05.446612 3516 webserver.cc:460] Webserver started at http://127.2.180.254:38453/ using document root <none> and password file <none>
I20260808 15:08:05.447230 3516 fs_manager.cc:359] Metadata directory not provided
I20260808 15:08:05.447335 3516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:08:05.447642 3516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:08:05.450443 3516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1"
format_stamp: "Formatted at 2026-08-08 15:08:05 on dist-test-slave-jlzn"
server_key: "b53fbd045d012512e226b51d89244126"
server_key_iv: "a2ae6e30eb71177a651027886331d3af"
server_key_version: "encryptionkey@0"
I20260808 15:08:05.451193 3516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1"
format_stamp: "Formatted at 2026-08-08 15:08:05 on dist-test-slave-jlzn"
server_key: "b53fbd045d012512e226b51d89244126"
server_key_iv: "a2ae6e30eb71177a651027886331d3af"
server_key_version: "encryptionkey@0"
I20260808 15:08:05.455986 3516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 15:08:05.459079 3536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:08:05.460323 3516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:08:05.460482 3516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1"
format_stamp: "Formatted at 2026-08-08 15:08:05 on dist-test-slave-jlzn"
server_key: "b53fbd045d012512e226b51d89244126"
server_key_iv: "a2ae6e30eb71177a651027886331d3af"
server_key_version: "encryptionkey@0"
I20260808 15:08:05.460708 3516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:08:05.473682 3516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:08:05.477531 3516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:08:05.477762 3516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:08:05.494431 3516 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:40683
I20260808 15:08:05.494449 3607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:40683 every 8 connection(s)
I20260808 15:08:05.495668 3516 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:08:05.499055 3608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:08:05.499346 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3516
I20260808 15:08:05.499576 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:08:05.500401 2771 external_mini_cluster.cc:1428] Setting key 9f15972e772b0f38c80c9f37a30e6b0c
I20260808 15:08:05.507191 3608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:05 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:08:05.527837 3612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.510441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60224 (local address 127.2.180.254:40683)
0808 15:08:05.512023 (+ 1582us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:05.512031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:05.512087 (+ 56us) server_negotiation.cc:407] Connection header received
0808 15:08:05.513575 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:05.513582 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:05.513787 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:05.513951 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:05.515187 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:05.516418 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:05.517742 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:05.518266 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:05.518425 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:05.521691 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:05.521699 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:05.521709 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:05.521737 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:05.524288 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:05.524918 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:05.524924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:05.524926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:05.525065 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:05.525427 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:05.525431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:05.525433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:05.525875 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:05.526058 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:05.526658 (+ 600us) server_negotiation.cc:299] Negotiation successful
0808 15:08:05.526880 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1377,"thread_start_us":1160,"threads_started":1}
I20260808 15:08:05.527902 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.510440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40683 (local address 127.0.0.1:60224)
0808 15:08:05.511204 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.511283 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:05.513240 (+ 1957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:05.513993 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:05.514000 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:05.514674 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:05.514984 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:05.515007 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:05.516569 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:05.516573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:05.517606 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:05.517619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:05.518488 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:05.518505 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:05.518665 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:05.519229 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:05.519250 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:05.521483 (+ 2233us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:05.524405 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:05.524417 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:05.524427 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:05.524800 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:05.525170 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:05.525175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:05.525179 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:05.525327 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:05.526023 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:05.526031 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:05.526248 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 15:08:05.526511 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 15:08:05.527090 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":97,"thread_start_us":316,"threads_started":1}
WARNING: no policy specified for kudu/127.2.180.253@KRBTEST.COM; defaulting to no policy
I20260808 15:08:05.530807 3608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:08:05.531831 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.531448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59558)
0808 15:08:05.531606 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.531714 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 15:08:05.532090 3537 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.253@KRBTEST.COM" created.
I20260808 15:08:05.533564 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.533182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59572)
0808 15:08:05.533331 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.533413 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 15:08:05.533773 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:42213: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab.
Entry for principal kudu/127.2.180.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab.
I20260808 15:08:05.560928 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.253:42213
--webserver_interface=127.2.180.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab
--principal=kudu/127.2.180.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:37791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40683,127.2.180.253:42213,127.2.180.252:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:08:05.583092 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } attempt: 1
I20260808 15:08:05.584200 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.583768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59578)
0808 15:08:05.583958 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.584076 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 15:08:05.585140 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.584769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59592)
0808 15:08:05.584913 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.584984 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:08:05.585327 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:42213: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
I20260808 15:08:05.653625 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } attempt: 2
I20260808 15:08:05.654719 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.654297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59594)
0808 15:08:05.654483 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.654570 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 15:08:05.655755 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.655399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59598)
0808 15:08:05.655523 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.655607 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:08:05.655968 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:42213: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
I20260808 15:08:05.747180 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } attempt: 3
I20260808 15:08:05.748420 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.747999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59602)
0808 15:08:05.748190 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.748282 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 15:08:05.749476 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.749131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59616)
0808 15:08:05.749273 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.749342 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 15:08:05.749706 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:42213: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
W20260808 15:08:05.879864 3616 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:08:05.880182 3616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:08:05.880270 3616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:08:05.880352 3616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:08:05.887315 3616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:08:05.887398 3616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:08:05.887459 3616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:08:05.887508 3616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:08:05.887576 3616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:08:05.892918 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } attempt: 4
I20260808 15:08:05.894054 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.893538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59626)
0808 15:08:05.893834 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.893923 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260808 15:08:05.894984 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.894677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59640)
0808 15:08:05.894786 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.894863 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 15:08:05.895143 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:42213: Network error: Client connection negotiation failed: client connection to 127.2.180.253:42213: connect: Connection refused (error 111)
I20260808 15:08:05.897575 3616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:37791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40683,127.2.180.253:42213,127.2.180.252:44873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab
--principal=kudu/127.2.180.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.253:42213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.180.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:08:05.898718 3616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:08:05.900032 3616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:08:05.911015 3622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:05.911091 3621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:05.911079 3627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:08:05.911880 3616 server_base.cc:1034] running on GCE node
I20260808 15:08:05.912636 3616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:08:05.913844 3616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:08:05.915076 3616 hybrid_clock.cc:648] HybridClock initialized: now 1786201685914999 us; error 33 us; skew 500 ppm
I20260808 15:08:05.915369 3616 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:08:05 dist-test-slave-jlzn krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:08:05.918315 3616 init.cc:377] Logged in from keytab as kudu/127.2.180.253@KRBTEST.COM (short username kudu)
I20260808 15:08:05.919571 3616 webserver.cc:460] Webserver started at http://127.2.180.253:42135/ using document root <none> and password file <none>
I20260808 15:08:05.920262 3616 fs_manager.cc:359] Metadata directory not provided
I20260808 15:08:05.920368 3616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:08:05.920661 3616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:08:05.923398 3616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/data/instance:
uuid: "eb819a28217341f592861a824cc080eb"
format_stamp: "Formatted at 2026-08-08 15:08:05 on dist-test-slave-jlzn"
server_key: "053e52aee5840a572cd05744a6ab29cd"
server_key_iv: "ed1aac3dfefe574fd84a76a643861851"
server_key_version: "encryptionkey@0"
I20260808 15:08:05.924182 3616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/wal/instance:
uuid: "eb819a28217341f592861a824cc080eb"
format_stamp: "Formatted at 2026-08-08 15:08:05 on dist-test-slave-jlzn"
server_key: "053e52aee5840a572cd05744a6ab29cd"
server_key_iv: "ed1aac3dfefe574fd84a76a643861851"
server_key_version: "encryptionkey@0"
I20260808 15:08:05.928803 3616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 15:08:05.931989 3638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:08:05.933200 3616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 15:08:05.933363 3616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/wal
uuid: "eb819a28217341f592861a824cc080eb"
format_stamp: "Formatted at 2026-08-08 15:08:05 on dist-test-slave-jlzn"
server_key: "053e52aee5840a572cd05744a6ab29cd"
server_key_iv: "ed1aac3dfefe574fd84a76a643861851"
server_key_version: "encryptionkey@0"
I20260808 15:08:05.933547 3616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:08:05.948467 3616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:08:05.952350 3616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:08:05.952580 3616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:08:05.969959 3616 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.253:42213
I20260808 15:08:05.969976 3709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.253:42213 every 8 connection(s)
I20260808 15:08:05.971133 3616 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
I20260808 15:08:05.971935 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3616
I20260808 15:08:05.972100 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-1/wal/instance
I20260808 15:08:05.972501 2771 external_mini_cluster.cc:1428] Setting key 2f147884cfae207d06fa7d6e8c8103e7
I20260808 15:08:05.975945 3711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:08:05.982179 3711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:05 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.2.180.252@KRBTEST.COM; defaulting to no policy
I20260808 15:08:06.001677 3612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.984484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.2.180.254:40683)
0808 15:08:05.984695 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:05.984701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:05.985842 (+ 1141us) server_negotiation.cc:407] Connection header received
0808 15:08:05.987621 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:05.987628 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:05.987742 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:05.987901 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:05.989249 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:05.990236 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:05.991688 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:05.992319 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:05.992445 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:05.995626 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:05.995633 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:05.995638 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:05.995684 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:05.998875 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:05.999554 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:05.999561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:05.999565 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:05.999729 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.000143 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.000148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.000151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.000501 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.000645 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.001242 (+ 597us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.001407 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 15:08:06.002585 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:05.984938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40683 (local address 127.0.0.1:60238)
0808 15:08:05.985667 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:05.985718 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:05.987363 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:05.988061 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:05.988069 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:05.988534 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:05.989080 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:05.989097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:05.990367 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:05.990371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:05.991525 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:05.991537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:05.992505 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:05.992511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:05.992727 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:05.993335 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:05.993357 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:05.995377 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:05.999009 (+ 3632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:05.999015 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:05.999025 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:05.999438 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:05.999857 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:05.999861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:05.999863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.000038 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.000650 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:06.000656 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:06.000931 (+ 275us) client_negotiation.cc:769] Sending connection context
0808 15:08:06.001145 (+ 214us) client_negotiation.cc:240] Negotiation successful
0808 15:08:06.001486 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":298,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.252@KRBTEST.COM" created.
I20260808 15:08:06.004307 3711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:08:06.018853 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.005031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59650)
0808 15:08:06.005211 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.005232 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:06.005324 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:06.006396 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:06.006399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:06.006740 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:06.006870 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.006877 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.008622 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.008626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.009446 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.009456 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.010434 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.010438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.010551 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.011041 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:06.011057 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:06.012938 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:06.016344 (+ 3406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.016350 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.016354 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.016832 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.017169 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.017173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.017176 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.017337 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.017899 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:06.017905 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:06.018037 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 15:08:06.018175 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 15:08:06.018438 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 15:08:06.018888 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.005281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59650 (local address 127.2.180.253:42213)
0808 15:08:06.005936 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.005944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.005972 (+ 28us) server_negotiation.cc:407] Connection header received
0808 15:08:06.006021 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.006028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.006230 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.006372 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.007034 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.008493 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.009582 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.010293 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.010407 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.013149 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.013162 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.013176 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.013210 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.016224 (+ 3014us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.016943 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.016950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.016952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.017069 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.017447 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.017450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.017452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.017779 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.017934 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.018260 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.018538 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":492,"thread_start_us":343,"threads_started":1}
I20260808 15:08:06.021133 3711 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:08:06.022118 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.021718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43638)
0808 15:08:06.021862 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.021970 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:08:06.022329 3640 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
I20260808 15:08:06.023725 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.023424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43642)
0808 15:08:06.023554 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.023632 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 15:08:06.023952 3711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab.
Entry for principal kudu/127.2.180.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab.
I20260808 15:08:06.038635 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.252:44873
--webserver_interface=127.2.180.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab
--principal=kudu/127.2.180.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:37791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:40683,127.2.180.253:42213,127.2.180.252:44873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:08:06.073267 3711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 1
I20260808 15:08:06.074393 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.073943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43646)
0808 15:08:06.074144 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.074242 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 15:08:06.075347 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.075024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43650)
0808 15:08:06.075165 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.075241 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 15:08:06.075553 3711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
I20260808 15:08:06.143749 3711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 2
I20260808 15:08:06.144899 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.144454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43658)
0808 15:08:06.144648 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.144740 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 15:08:06.145838 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.145544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43668)
0808 15:08:06.145661 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.145729 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 15:08:06.146045 3711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
I20260808 15:08:06.194404 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } attempt: 5
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:08:06.207427 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.195189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59654)
0808 15:08:06.195381 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.195404 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:06.195536 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:06.195901 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:06.195906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:06.196359 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:06.196561 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.196572 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.197549 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.197555 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.198418 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.198431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.199175 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.199182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.199393 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.200090 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:06.200109 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:06.202308 (+ 2199us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:06.205141 (+ 2833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.205147 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.205151 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.205518 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.205856 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.205861 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.205865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.205999 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.206498 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:06.206503 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:06.206651 (+ 148us) client_negotiation.cc:769] Sending connection context
0808 15:08:06.206798 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:08:06.207071 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 15:08:06.207427 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.195331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.2.180.253:42213)
0808 15:08:06.195518 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.195525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.195578 (+ 53us) server_negotiation.cc:407] Connection header received
0808 15:08:06.195647 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.195652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.195735 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.195842 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.196725 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.197411 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.198574 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.199029 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.199143 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.202486 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.202491 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.202494 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.202527 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.205011 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.205633 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.205637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.205639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.205755 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.206102 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.206107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.206110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.206385 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.206515 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.206909 (+ 394us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.207130 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 15:08:06.208547 3608 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:08:06.209553 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.209195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43684)
0808 15:08:06.209344 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.209416 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 15:08:06.210512 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.210229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43696)
0808 15:08:06.210343 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.210421 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 15:08:06.210750 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
I20260808 15:08:06.237229 3711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 3
I20260808 15:08:06.238183 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.237751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43710)
0808 15:08:06.237931 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.238022 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 15:08:06.239140 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.238746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43724)
0808 15:08:06.238945 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.239017 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260808 15:08:06.239331 3711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
I20260808 15:08:06.261940 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 1
I20260808 15:08:06.262921 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.262503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43740)
0808 15:08:06.262687 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.262768 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 15:08:06.263835 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.263503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43754)
0808 15:08:06.263643 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.263708 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 15:08:06.264073 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
I20260808 15:08:06.332278 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 2
I20260808 15:08:06.333384 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.332896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43758)
0808 15:08:06.333128 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.333218 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 15:08:06.334324 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.333996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43762)
0808 15:08:06.334143 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.334210 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 15:08:06.334523 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
W20260808 15:08:06.360800 3720 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:08:06.361099 3720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:08:06.361189 3720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:08:06.361297 3720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:08:06.368242 3720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:08:06.368322 3720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:08:06.368409 3720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:08:06.368471 3720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:08:06.368563 3720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:08:06.377735 3720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:37791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:40683,127.2.180.253:42213,127.2.180.252:44873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab
--principal=kudu/127.2.180.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.252:44873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.180.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:08:06.378885 3720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:08:06.380126 3720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:08:06.382509 3711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 4
I20260808 15:08:06.383517 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.383070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43774)
0808 15:08:06.383280 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.383363 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 15:08:06.384451 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.384130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43790)
0808 15:08:06.384275 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.384344 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:08:06.384719 3711 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
W20260808 15:08:06.390892 3726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:06.390941 3737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:06.390964 3729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:08:06.391641 3720 server_base.cc:1034] running on GCE node
I20260808 15:08:06.392356 3720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:08:06.393555 3720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:08:06.394791 3720 hybrid_clock.cc:648] HybridClock initialized: now 1786201686394725 us; error 27 us; skew 500 ppm
I20260808 15:08:06.395097 3720 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201686, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:08:06.398102 3720 init.cc:377] Logged in from keytab as kudu/127.2.180.252@KRBTEST.COM (short username kudu)
I20260808 15:08:06.399377 3720 webserver.cc:460] Webserver started at http://127.2.180.252:38531/ using document root <none> and password file <none>
I20260808 15:08:06.400034 3720 fs_manager.cc:359] Metadata directory not provided
I20260808 15:08:06.400141 3720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:08:06.400429 3720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:08:06.403213 3720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/data/instance:
uuid: "5bc8ff249cdd476982a60d832bc9a258"
format_stamp: "Formatted at 2026-08-08 15:08:06 on dist-test-slave-jlzn"
server_key: "be11644f1d5186f79b0eb0c277cd7841"
server_key_iv: "96ccff76cb91bb96cfc059e28e6540ca"
server_key_version: "encryptionkey@0"
I20260808 15:08:06.404057 3720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/wal/instance:
uuid: "5bc8ff249cdd476982a60d832bc9a258"
format_stamp: "Formatted at 2026-08-08 15:08:06 on dist-test-slave-jlzn"
server_key: "be11644f1d5186f79b0eb0c277cd7841"
server_key_iv: "96ccff76cb91bb96cfc059e28e6540ca"
server_key_version: "encryptionkey@0"
I20260808 15:08:06.409054 3720 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260808 15:08:06.412197 3746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:08:06.413407 3720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260808 15:08:06.413585 3720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/wal
uuid: "5bc8ff249cdd476982a60d832bc9a258"
format_stamp: "Formatted at 2026-08-08 15:08:06 on dist-test-slave-jlzn"
server_key: "be11644f1d5186f79b0eb0c277cd7841"
server_key_iv: "96ccff76cb91bb96cfc059e28e6540ca"
server_key_version: "encryptionkey@0"
I20260808 15:08:06.413776 3720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:08:06.434267 3720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:08:06.438134 3720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:08:06.438359 3720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:08:06.440708 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 3
I20260808 15:08:06.441645 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.441216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43800)
0808 15:08:06.441392 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.441477 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 15:08:06.442833 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.442309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43816)
0808 15:08:06.442560 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.442683 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260808 15:08:06.443089 3608 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44873: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44873: connect: Connection refused (error 111)
I20260808 15:08:06.455874 3720 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.252:44873
I20260808 15:08:06.455931 3822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.252:44873 every 8 connection(s)
I20260808 15:08:06.457055 3720 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
I20260808 15:08:06.460518 3823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:08:06.463222 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 3720
I20260808 15:08:06.463382 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786201338129629-2771-0/minicluster-data/master-2/wal/instance
I20260808 15:08:06.463855 2771 external_mini_cluster.cc:1428] Setting key 943b4e65377bacddb1249ae85de7526b
I20260808 15:08:06.467561 3823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201686, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:08:06.485321 3612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.471135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60250 (local address 127.2.180.254:40683)
0808 15:08:06.471314 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.471320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.472076 (+ 756us) server_negotiation.cc:407] Connection header received
0808 15:08:06.472243 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.472247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.472307 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.472382 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.473538 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.474191 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.475624 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.476102 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.476211 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.479468 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.479473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.479476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.479512 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.482482 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.483237 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.483241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.483243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.483367 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.483774 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.483778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.483781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.484144 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.484300 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.484920 (+ 620us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.485076 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:08:06.485558 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.465663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43824 (local address 127.2.180.252:44873)
0808 15:08:06.466409 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.466424 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.466780 (+ 356us) server_negotiation.cc:407] Connection header received
0808 15:08:06.468659 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.468685 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.469164 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.469583 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.470769 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.472709 (+ 1940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.473983 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.474609 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.474787 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.478702 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.478712 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.478723 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.478752 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.481583 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.482303 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.482309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.482315 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.482458 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.482834 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.482839 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.482842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.483411 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.483688 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.484010 (+ 322us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.484305 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":527,"thread_start_us":308,"threads_started":1}
I20260808 15:08:06.485591 3829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.471186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:40683 (local address 127.0.0.1:60250)
0808 15:08:06.471929 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.471968 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:06.472139 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:06.472451 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:06.472457 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:06.473153 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:06.473387 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.473396 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.474347 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.474351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.475489 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.475503 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.476319 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.476326 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.476461 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.476977 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:06.477011 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:06.479287 (+ 2276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:06.482598 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.482607 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.482612 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.483120 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.483467 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.483471 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.483475 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.483658 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.484451 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:06.484459 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:06.484640 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 15:08:06.484843 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 15:08:06.485147 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":331,"threads_started":1}
I20260808 15:08:06.487133 3823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201686, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:08:06.499768 3829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.487696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:42213 (local address 127.0.0.1:59656)
0808 15:08:06.487863 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.487880 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:06.487972 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:06.488256 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:06.488259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:06.488584 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:06.488764 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.488776 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.489762 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.489767 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.490587 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.490598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.491273 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.491278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.491439 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.491951 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:06.491968 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:06.494004 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:06.497312 (+ 3308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.497318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.497322 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.497712 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.498114 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.498119 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.498122 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.498289 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.498886 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:06.498892 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:06.499059 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 15:08:06.499209 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 15:08:06.499452 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 15:08:06.499902 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.487854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59656 (local address 127.2.180.253:42213)
0808 15:08:06.488003 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.488007 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.488022 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:08:06.488079 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.488082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.488137 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.488227 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.488962 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.489630 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.490725 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.491136 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.491251 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.494188 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.494194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.494199 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.494238 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.497197 (+ 2959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.497830 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.497836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.497839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.497995 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.498399 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.498404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.498408 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.498745 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.498891 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.499312 (+ 421us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.499521 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 15:08:06.500916 3823 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:08:06.504163 3612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.488796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60258 (local address 127.2.180.254:40683)
0808 15:08:06.488938 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.488942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.488978 (+ 36us) server_negotiation.cc:407] Connection header received
0808 15:08:06.489159 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.489164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.489224 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.489300 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.490374 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.491081 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.492389 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.492799 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.492925 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.496911 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.496917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.496922 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.496957 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.500743 (+ 3786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.501401 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.501407 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.501410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.501598 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.502027 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.502030 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.502032 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.502376 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.502646 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.503537 (+ 891us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.503763 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201686, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:08:06.514137 3829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.501549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43826)
0808 15:08:06.501763 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.501780 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:06.501862 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:06.502162 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:06.502166 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:06.502585 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:06.502747 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.502755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.503733 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.503736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.504724 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.504735 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.505808 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.505814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.505984 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.506447 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:06.506469 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:06.508681 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:06.511987 (+ 3306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.511990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.511993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.512336 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.512701 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.512704 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.512707 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.512826 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.513362 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:06.513368 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:06.513526 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 15:08:06.513674 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 15:08:06.513886 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 15:08:06.514394 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.501649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.2.180.252:44873)
0808 15:08:06.501796 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.501801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.501818 (+ 17us) server_negotiation.cc:407] Connection header received
0808 15:08:06.501958 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.501963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.502024 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.502118 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.502863 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.503609 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.504857 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.505359 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.505483 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.508857 (+ 3374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.508863 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.508868 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.508903 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.511865 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.512446 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.512451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.512455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.512588 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.512930 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.512935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.512938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.513239 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.513343 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.513785 (+ 442us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.514047 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":55}
I20260808 15:08:06.519076 3823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: Bootstrap starting.
I20260808 15:08:06.519977 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.505665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59664 (local address 127.2.180.253:42213)
0808 15:08:06.505813 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.505816 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.505845 (+ 29us) server_negotiation.cc:407] Connection header received
0808 15:08:06.505881 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.505884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.505931 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.505993 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.507496 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.508271 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.509714 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.510117 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.510216 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.514792 (+ 4576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.514799 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.514803 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.514838 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.517495 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.518227 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.518232 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.518236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.518366 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.518719 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.518723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.518725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.518961 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.519190 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.519459 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.519672 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 15:08:06.522746 3823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: Neither blocks nor log segments found. Creating new log.
I20260808 15:08:06.523910 3823 log.cc:824] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: Log is configured to *not* fsync() on all Append() calls
I20260808 15:08:06.524624 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:08:06.526574 3823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: No bootstrap required, opened a new log
I20260808 15:08:06.527257 3573 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60258
I20260808 15:08:06.536859 3823 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } }
I20260808 15:08:06.537158 3823 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bc8ff249cdd476982a60d832bc9a258, State: Initialized, Role: FOLLOWER
I20260808 15:08:06.537609 3823 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } }
I20260808 15:08:06.538394 3832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.538700 3832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:06.539600 3823 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 15:08:06.548219 3850 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:08:06.548346 3850 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:08:06.546115 3720 master_runner.cc:186] Error getting master registration for 127.2.180.254:40683: OK, instance_id { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" instance_seqno: 1786201685489698 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:08:06.557229 3720 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:08:06.620312 3608 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 4
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
I20260808 15:08:06.635102 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.621157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43830 (local address 127.2.180.252:44873)
0808 15:08:06.621354 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.621360 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.621439 (+ 79us) server_negotiation.cc:407] Connection header received
0808 15:08:06.621628 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.621634 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.621719 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.621820 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.622808 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.623991 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.625789 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.626512 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.626645 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.629756 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.629762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.629765 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.629802 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.632478 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.633106 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.633111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.633113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.633235 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.633630 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.633633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.633636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.633964 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.634121 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.634538 (+ 417us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.634754 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 15:08:06.635145 3611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.621096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43830)
0808 15:08:06.621314 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.621335 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:06.621484 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:06.621870 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:06.621875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:06.622419 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:06.622644 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.622657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.624130 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.624135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.625632 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.625647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.626653 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.626659 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.626848 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.627452 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:06.627470 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:06.629585 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:06.632603 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.632608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.632613 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.632991 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.633341 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.633347 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.633351 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.633497 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.634114 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:06.634119 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:06.634287 (+ 168us) client_negotiation.cc:769] Sending connection context
0808 15:08:06.634409 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 15:08:06.634698 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260808 15:08:06.641170 3608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1: Bootstrap starting.
I20260808 15:08:06.645299 3608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1: Neither blocks nor log segments found. Creating new log.
I20260808 15:08:06.646605 3608 log.cc:824] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1: Log is configured to *not* fsync() on all Append() calls
I20260808 15:08:06.649865 3608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1: No bootstrap required, opened a new log
I20260808 15:08:06.657229 3608 raft_consensus.cc:348] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } }
I20260808 15:08:06.657610 3608 raft_consensus.cc:729] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d39e34bbcc2b4e1b9b4a0bb914767ed1, State: Initialized, Role: FOLLOWER
I20260808 15:08:06.658103 3608 consensus_queue.cc:260] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } }
I20260808 15:08:06.659052 3855 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.659432 3855 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:06.660226 3608 sys_catalog.cc:564] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:08:06.660343 3573 raft_consensus.cc:482] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 15:08:06.660529 3573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:08:06.663431 3573 raft_consensus.cc:504] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } }
I20260808 15:08:06.666034 3573 leader_election.cc:288] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [CANDIDATE]: Term 1 election: Requested vote from peers eb819a28217341f592861a824cc080eb (127.2.180.253:42213), 5bc8ff249cdd476982a60d832bc9a258 (127.2.180.252:44873)
I20260808 15:08:06.666178 3780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5bc8ff249cdd476982a60d832bc9a258"
I20260808 15:08:06.666682 3780 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:08:06.669823 3780 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d39e34bbcc2b4e1b9b4a0bb914767ed1 in term 1.
I20260808 15:08:06.671092 3538 leader_election.cc:302] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bc8ff249cdd476982a60d832bc9a258, d39e34bbcc2b4e1b9b4a0bb914767ed1; no voters:
I20260808 15:08:06.671689 3855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [term 1 FOLLOWER]: Leader election won for term 1
W20260808 15:08:06.675964 3872 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:08:06.676190 3872 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:08:06.683931 3711 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } attempt: 5
I20260808 15:08:06.686674 3855 raft_consensus.cc:686] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [term 1 LEADER]: Becoming Leader. State: Replica: d39e34bbcc2b4e1b9b4a0bb914767ed1, State: Running, Role: LEADER
I20260808 15:08:06.687202 3855 consensus_queue.cc:237] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } }
W20260808 15:08:06.690219 3754 tablet.cc:2369] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 15:08:06 dist-test-slave-jlzn krb5kdc[3497](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201685, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
I20260808 15:08:06.693531 3876 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d39e34bbcc2b4e1b9b4a0bb914767ed1. Latest consensus state: current_term: 1 leader_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.693892 3876 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:06.704435 3882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:08:06.708757 3716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.684531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44873 (local address 127.0.0.1:43840)
0808 15:08:06.684905 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:06.684925 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:06.685038 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:06.685344 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:06.685349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:06.685864 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:06.686074 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.686087 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.687368 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.687373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.688726 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:06.688742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.689900 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.689907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:06.690157 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.690828 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:06.690851 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:06.693641 (+ 2790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:06.705381 (+ 11740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.705388 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.705392 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.705977 (+ 585us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.706835 (+ 858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:06.706840 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:06.706844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:06.707026 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:06.707726 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:06.707731 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:06.707927 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 15:08:06.708096 (+ 169us) client_negotiation.cc:240] Negotiation successful
0808 15:08:06.708384 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233}
I20260808 15:08:06.709756 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.684858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43840 (local address 127.2.180.252:44873)
0808 15:08:06.685006 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.685011 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.685029 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:08:06.685147 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.685152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.685224 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.685305 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.686217 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.687241 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.688870 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.689580 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.689700 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.694204 (+ 4504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.694213 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.694218 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.694262 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.705032 (+ 10770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.706476 (+ 1444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.706482 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.706486 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.706685 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.707172 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.707177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.707180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.707513 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.707658 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.709187 (+ 1529us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.709419 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 15:08:06.715317 3711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb: Bootstrap starting.
I20260808 15:08:06.720125 3711 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb: Neither blocks nor log segments found. Creating new log.
I20260808 15:08:06.721635 3882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:08:06.722990 3711 log.cc:824] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb: Log is configured to *not* fsync() on all Append() calls
I20260808 15:08:06.697324 3516 master_runner.cc:186] Error getting master registration for 127.2.180.253:42213: OK, instance_id { permanent_uuid: "eb819a28217341f592861a824cc080eb" instance_seqno: 1786201685965249 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:08:06.725710 3516 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 15:08:06.730187 3542 tablet.cc:2369] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:08:06.738555 3711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb: No bootstrap required, opened a new log
I20260808 15:08:06.742362 3780 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [term 1 FOLLOWER]: Refusing update from remote peer d39e34bbcc2b4e1b9b4a0bb914767ed1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:08:06.743516 3855 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:08:06.753104 3828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.706678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43846 (local address 127.2.180.252:44873)
0808 15:08:06.717683 (+ 11005us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.717689 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.720814 (+ 3125us) server_negotiation.cc:407] Connection header received
0808 15:08:06.720960 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.720964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.721097 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.721236 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.724744 (+ 3508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.726549 (+ 1805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.729782 (+ 3233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.730832 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.731042 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.741098 (+ 10056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.741107 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.741112 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.741152 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.749507 (+ 8355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.750267 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.750273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.750277 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.750432 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.750855 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.750860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.750863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.751226 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.751491 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.752450 (+ 959us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.752676 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10884,"thread_start_us":12385,"threads_started":1}
I20260808 15:08:06.764040 3711 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } }
I20260808 15:08:06.764523 3711 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb819a28217341f592861a824cc080eb, State: Initialized, Role: FOLLOWER
I20260808 15:08:06.765097 3711 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } }
I20260808 15:08:06.767957 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.701849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.2.180.253:42213)
0808 15:08:06.702024 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.702030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.718166 (+ 16136us) server_negotiation.cc:407] Connection header received
0808 15:08:06.718350 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.718356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.718428 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.718518 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.734417 (+ 15899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.735498 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.742437 (+ 6939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.743224 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.743326 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.754584 (+ 11258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.754591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.754597 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.754635 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.758140 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.763565 (+ 5425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.763571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.763575 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.763726 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.764207 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.764213 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.764217 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.764656 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.764890 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.767330 (+ 2440us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.767577 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:08:06.770529 3612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:06.701468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.2.180.254:40683)
0808 15:08:06.701632 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:06.701638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:06.705933 (+ 4295us) server_negotiation.cc:407] Connection header received
0808 15:08:06.706130 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:06.706135 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:06.706207 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:06.706295 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:06.736190 (+ 29895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.737391 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.743921 (+ 6530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:06.744602 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:06.744704 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:06.759090 (+ 14386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:06.759097 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:06.759102 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:06.759144 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:06.762718 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.765363 (+ 2645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.765368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.765371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.765542 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:06.765976 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:06.765981 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:06.765984 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:06.766353 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:06.766697 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:06.769899 (+ 3202us) server_negotiation.cc:299] Negotiation successful
0808 15:08:06.770145 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 15:08:06.773159 3832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d39e34bbcc2b4e1b9b4a0bb914767ed1. Latest consensus state: current_term: 1 leader_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.773455 3832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:06.773128 3895 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.775370 3711 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:08:06.775825 3675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:08:06.775446 3895 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:06.776525 3676 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eb819a28217341f592861a824cc080eb"
I20260808 15:08:06.779994 3675 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [term 1 FOLLOWER]: Refusing update from remote peer d39e34bbcc2b4e1b9b4a0bb914767ed1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:08:06.783682 3676 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d39e34bbcc2b4e1b9b4a0bb914767ed1 in term 1.
I20260808 15:08:06.786190 3855 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:08:06.793249 3889 mvcc.cc:205] Tried to move back new op lower bound from 7316282108883009536 to 7316282108675538944. Current Snapshot: MvccSnapshot[applied={T|T < 7316282108883009536}]
I20260808 15:08:06.810220 3890 mvcc.cc:205] Tried to move back new op lower bound from 7316282108883009536 to 7316282108675538944. Current Snapshot: MvccSnapshot[applied={T|T < 7316282108883009536}]
I20260808 15:08:06.814713 3895 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d39e34bbcc2b4e1b9b4a0bb914767ed1. Latest consensus state: current_term: 1 leader_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.815032 3895 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:06.817955 3895 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.818193 3895 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:06.822022 3876 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.822294 3876 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:06.822721 3876 sys_catalog.cc:455] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.822933 3876 sys_catalog.cc:458] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1 [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:06.826504 3882 catalog_manager.cc:1347] Generated new cluster ID: e4c3ba839c9d4378b895dcdccf802731
I20260808 15:08:06.826610 3882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:08:06.827035 3832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d39e34bbcc2b4e1b9b4a0bb914767ed1" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 40683 } } peers { permanent_uuid: "eb819a28217341f592861a824cc080eb" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 42213 } } peers { permanent_uuid: "5bc8ff249cdd476982a60d832bc9a258" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44873 } } }
I20260808 15:08:06.827298 3832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:06.848925 3920 catalog_manager.cc:1259] Loaded cluster ID: e4c3ba839c9d4378b895dcdccf802731
I20260808 15:08:06.849083 3920 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb: loading cluster ID for follower catalog manager: success
W20260808 15:08:06.850309 3920 catalog_manager.cc:1576] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260808 15:08:06.850507 3920 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:08:06.865345 3882 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:08:06.867053 3882 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:08:06.894928 3882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d39e34bbcc2b4e1b9b4a0bb914767ed1: Generated new TSK 0
I20260808 15:08:06.895948 3882 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 15:08:07.037622 3612 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.013337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60266 (local address 127.2.180.254:40683)
0808 15:08:07.013541 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.013548 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.035362 (+ 21814us) server_negotiation.cc:407] Connection header received
0808 15:08:07.035890 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.035897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.036007 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.036105 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.037355 (+ 1250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60266: BlockingRecv error: recv got EOF from 127.0.0.1:60266 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260808 15:08:07.040125 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.019961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59682 (local address 127.2.180.253:42213)
0808 15:08:07.020169 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.020174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.038248 (+ 18074us) server_negotiation.cc:407] Connection header received
0808 15:08:07.038776 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.038783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.038888 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.038965 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.039896 (+ 931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59682: BlockingRecv error: recv got EOF from 127.0.0.1:59682 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 15:08:07.043558 3828 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.041215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43854 (local address 127.2.180.252:44873)
0808 15:08:07.041393 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.041399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.041953 (+ 554us) server_negotiation.cc:407] Connection header received
0808 15:08:07.042222 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.042227 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.042305 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.042381 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.043301 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43854: BlockingRecv error: recv got EOF from 127.0.0.1:43854 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 15:08:07.087602 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.074686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59698 (local address 127.2.180.253:42213)
0808 15:08:07.074861 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.074867 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.085397 (+ 10530us) server_negotiation.cc:407] Connection header received
0808 15:08:07.085924 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.085931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.086090 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.086176 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.087275 (+ 1099us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59698: BlockingRecv error: recv got EOF from 127.0.0.1:59698 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 15:08:07.135340 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.122632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59714 (local address 127.2.180.253:42213)
0808 15:08:07.122808 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.122814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.133356 (+ 10542us) server_negotiation.cc:407] Connection header received
0808 15:08:07.133840 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.133847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.133940 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.134033 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.135098 (+ 1065us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59714: BlockingRecv error: recv got EOF from 127.0.0.1:59714 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 15:08:07.166236 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.163959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.2.180.253:42213)
0808 15:08:07.164180 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.164187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.164759 (+ 572us) server_negotiation.cc:407] Connection header received
0808 15:08:07.165024 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.165030 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.165126 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.165214 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.165977 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59722: BlockingRecv error: recv got EOF from 127.0.0.1:59722 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 15:08:07.175323 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.173552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59738 (local address 127.2.180.253:42213)
0808 15:08:07.173773 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.173779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.174065 (+ 286us) server_negotiation.cc:407] Connection header received
0808 15:08:07.174247 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.174252 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.174344 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.174425 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.175143 (+ 718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59738: BlockingRecv error: recv got EOF from 127.0.0.1:59738 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
W20260808 15:08:07.188520 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.186531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.2.180.253:42213)
0808 15:08:07.186711 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.186717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.187039 (+ 322us) server_negotiation.cc:407] Connection header received
0808 15:08:07.187298 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.187304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.187373 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.187425 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.188293 (+ 868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59748: BlockingRecv error: recv got EOF from 127.0.0.1:59748 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 15:08:07.209354 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.207375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59756 (local address 127.2.180.253:42213)
0808 15:08:07.207555 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.207561 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.208042 (+ 481us) server_negotiation.cc:407] Connection header received
0808 15:08:07.208222 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.208226 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.208292 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.208377 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.209105 (+ 728us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59756: BlockingRecv error: recv got EOF from 127.0.0.1:59756 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 15:08:07.251324 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.249501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59770 (local address 127.2.180.253:42213)
0808 15:08:07.249711 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.249717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.250033 (+ 316us) server_negotiation.cc:407] Connection header received
0808 15:08:07.250183 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.250189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.250283 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.250424 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.251058 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59770: BlockingRecv error: recv got EOF from 127.0.0.1:59770 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 15:08:07.320366 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.318480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.2.180.253:42213)
0808 15:08:07.318679 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.318685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.319036 (+ 351us) server_negotiation.cc:407] Connection header received
0808 15:08:07.319199 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.319204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.319295 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.319401 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.320151 (+ 750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59776: BlockingRecv error: recv got EOF from 127.0.0.1:59776 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 15:08:07.453361 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.451481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.2.180.253:42213)
0808 15:08:07.451675 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.451680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.452078 (+ 398us) server_negotiation.cc:407] Connection header received
0808 15:08:07.452269 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.452273 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.452346 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.452426 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.453109 (+ 683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59790: BlockingRecv error: recv got EOF from 127.0.0.1:59790 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 15:08:07.550195 3850 catalog_manager.cc:1259] Loaded cluster ID: e4c3ba839c9d4378b895dcdccf802731
I20260808 15:08:07.550297 3850 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: loading cluster ID for follower catalog manager: success
I20260808 15:08:07.552753 3850 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: acquiring CA information for follower catalog manager: success
I20260808 15:08:07.554440 3850 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5bc8ff249cdd476982a60d832bc9a258: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 15:08:07.714792 3717 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:07.713180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59804 (local address 127.2.180.253:42213)
0808 15:08:07.713419 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:07.713426 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:07.713606 (+ 180us) server_negotiation.cc:407] Connection header received
0808 15:08:07.713766 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:07.713770 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:07.713845 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:07.713930 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:07.714570 (+ 640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59804: BlockingRecv error: recv got EOF from 127.0.0.1:59804 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 15:08:07.853699 3920 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb: acquiring CA information for follower catalog manager: success
I20260808 15:08:07.855470 3920 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb819a28217341f592861a824cc080eb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:08:08.235538 4012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:08.230076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59814 (local address 127.2.180.253:42213)
0808 15:08:08.230771 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:08.230776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:08.230794 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:08:08.230859 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:08.230863 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:08.231030 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:08.231141 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:08.231546 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.232320 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.233595 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.233994 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.234096 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:08.234491 (+ 395us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:08.234998 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:08.235125 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:08.235172 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 15:08:08.235237 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":466,"threads_started":1}
I20260808 15:08:08.246279 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:08.241277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43868 (local address 127.2.180.252:44873)
0808 15:08:08.241728 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:08.241733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:08.241770 (+ 37us) server_negotiation.cc:407] Connection header received
0808 15:08:08.241962 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:08.241965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:08.242157 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:08.242291 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:08.242580 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.243251 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.244463 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.244851 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.244939 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:08.245313 (+ 374us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:08.245785 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:08.245886 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:08.245924 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 15:08:08.245984 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":239,"threads_started":1}
I20260808 15:08:08.259008 4012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:08.251753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59828 (local address 127.2.180.253:42213)
0808 15:08:08.251928 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:08.251934 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:08.253344 (+ 1410us) server_negotiation.cc:407] Connection header received
0808 15:08:08.253599 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:08.253603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:08.253682 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:08.253748 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:08.254522 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.255464 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.257408 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.257858 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.257972 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:08.258340 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:08.258436 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:08.258511 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:08.258684 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 15:08:08.258756 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 15:08:08.259028 4037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:08.251780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60282 (local address 127.2.180.254:40683)
0808 15:08:08.252272 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:08.252276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:08.252742 (+ 466us) server_negotiation.cc:407] Connection header received
0808 15:08:08.252971 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:08.252976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:08.253184 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:08.253359 (+ 175us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:08.254132 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.255200 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.256985 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.257377 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.257498 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:08.258002 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:08.258394 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:08.258500 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:08.258643 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 15:08:08.258725 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":275,"threads_started":1}
I20260808 15:08:08.260164 4027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:08.253477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.2.180.252:44873)
0808 15:08:08.253634 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:08.253640 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:08.254268 (+ 628us) server_negotiation.cc:407] Connection header received
0808 15:08:08.254465 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:08.254470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:08.254545 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:08.254641 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:08.255092 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.256108 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.258107 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:08.258685 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:08.258790 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:08.259485 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:08.259591 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:08.259653 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:08.259853 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 15:08:08.259924 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 15:08:08.263631 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3516
I20260808 15:08:08.265183 3599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:08:08.374596 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3516
W20260808 15:08:08.400170 3640 connection.cc:535] server connection from 127.0.0.1:59654 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 15:08:08.400549 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3616
I20260808 15:08:08.401602 3703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:08:08.507107 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3616
I20260808 15:08:08.532027 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 3720
I20260808 15:08:08.533278 3816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:08:08.636796 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 3720
2026-08-08T15:08:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5020 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 15:08:08 dist-test-slave-jlzn krb5kdc[4044](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 15:08:08 dist-test-slave-jlzn krb5kdc[4044](info): set up 2 sockets
Aug 08 15:08:08 dist-test-slave-jlzn krb5kdc[4044](info): commencing operation
krb5kdc: starting...
W20260808 15:08:09.835584 2771 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.003s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/test-user.keytab.
Aug 08 15:08:10 dist-test-slave-jlzn krb5kdc[4044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T15:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T15:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.2.180.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
Entry for principal kudu/127.2.180.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab.
I20260808 15:08:10.095243 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:33111
--webserver_interface=127.2.180.254
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:33111,127.2.180.253:41497,127.2.180.252:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 15:08:10.411944 4063 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:08:10.412267 4063 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:08:10.412362 4063 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:08:10.412474 4063 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:08:10.419497 4063 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:08:10.419591 4063 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:08:10.419684 4063 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:08:10.419749 4063 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:08:10.419853 4063 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:08:10.429272 4063 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:33111,127.2.180.253:41497,127.2.180.252:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.254.keytab
--principal=kudu/127.2.180.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.254:33111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.2.180.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:08:10.430449 4063 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:08:10.431699 4063 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:08:10.442641 4071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:10.442731 4070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:10.442677 4073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:08:10.443426 4063 server_base.cc:1034] running on GCE node
I20260808 15:08:10.444176 4063 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:08:10.445427 4063 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:08:10.446648 4063 hybrid_clock.cc:648] HybridClock initialized: now 1786201690446577 us; error 51 us; skew 500 ppm
I20260808 15:08:10.446959 4063 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:08:10 dist-test-slave-jlzn krb5kdc[4044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:08:10.454994 4063 init.cc:377] Logged in from keytab as kudu/127.2.180.254@KRBTEST.COM (short username kudu)
I20260808 15:08:10.457346 4063 webserver.cc:460] Webserver started at http://127.2.180.254:44495/ using document root <none> and password file <none>
I20260808 15:08:10.457978 4063 fs_manager.cc:359] Metadata directory not provided
I20260808 15:08:10.458076 4063 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:08:10.458374 4063 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:08:10.461266 4063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/data/instance:
uuid: "fa06706173f34959ae1186390853ffd9"
format_stamp: "Formatted at 2026-08-08 15:08:10 on dist-test-slave-jlzn"
server_key: "c09539319fc032e3a0b1b71968bf80d6"
server_key_iv: "c121f72e07f5ae5c037d524367db733f"
server_key_version: "encryptionkey@0"
I20260808 15:08:10.462023 4063 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/wal/instance:
uuid: "fa06706173f34959ae1186390853ffd9"
format_stamp: "Formatted at 2026-08-08 15:08:10 on dist-test-slave-jlzn"
server_key: "c09539319fc032e3a0b1b71968bf80d6"
server_key_iv: "c121f72e07f5ae5c037d524367db733f"
server_key_version: "encryptionkey@0"
I20260808 15:08:10.466850 4063 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260808 15:08:10.469993 4081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:08:10.471200 4063 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 15:08:10.471359 4063 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/wal
uuid: "fa06706173f34959ae1186390853ffd9"
format_stamp: "Formatted at 2026-08-08 15:08:10 on dist-test-slave-jlzn"
server_key: "c09539319fc032e3a0b1b71968bf80d6"
server_key_iv: "c121f72e07f5ae5c037d524367db733f"
server_key_version: "encryptionkey@0"
I20260808 15:08:10.471575 4063 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:08:10.494453 4063 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:08:10.498090 4063 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:08:10.498322 4063 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:08:10.515493 4063 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.254:33111
I20260808 15:08:10.515514 4153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.254:33111 every 8 connection(s)
I20260808 15:08:10.516752 4063 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/data/info.pb
I20260808 15:08:10.517231 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 4063
I20260808 15:08:10.517405 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-0/wal/instance
I20260808 15:08:10.517990 2771 external_mini_cluster.cc:1428] Setting key eabf131bb5ea18c98a9b9d334295aafc
I20260808 15:08:10.520473 4154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:08:10.526651 4154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:10 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:08:10.548794 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.529629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:33111 (local address 127.0.0.1:47722)
0808 15:08:10.530494 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.530544 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:10.532421 (+ 1877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:10.533295 (+ 874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:10.533300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:10.533836 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:10.534181 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:10.534199 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:10.536292 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:10.536296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:10.537336 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:10.537348 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:10.538510 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:10.538516 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:10.538644 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:10.539291 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:10.539312 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:10.541389 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:10.545100 (+ 3711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:10.545112 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:10.545123 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:10.545491 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:10.545960 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:10.545963 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:10.545966 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:10.546123 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:10.546987 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:10.546993 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:10.547190 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 15:08:10.547385 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 15:08:10.547757 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":353,"threads_started":1}
I20260808 15:08:10.548830 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.529627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.2.180.254:33111)
0808 15:08:10.531164 (+ 1537us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:10.531174 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:10.531231 (+ 57us) server_negotiation.cc:407] Connection header received
0808 15:08:10.532778 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:10.532784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:10.533077 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:10.533273 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:10.534392 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:10.536118 (+ 1726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:10.537478 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:10.538309 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:10.538475 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:10.541613 (+ 3138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:10.541622 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:10.541634 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:10.541673 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:10.544955 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:10.545641 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:10.545650 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:10.545654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:10.545852 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:10.546220 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:10.546225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:10.546228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:10.546862 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:10.547071 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:10.547563 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 15:08:10.547928 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":1321,"spinlock_wait_cycles":20736,"thread_start_us":1134,"threads_started":1}
WARNING: no policy specified for kudu/127.2.180.253@KRBTEST.COM; defaulting to no policy
I20260808 15:08:10.551441 4154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.253@KRBTEST.COM" created.
I20260808 15:08:10.552734 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.552179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42684)
0808 15:08:10.552402 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.552581 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 15:08:10.552990 4086 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
I20260808 15:08:10.554509 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.554120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42698)
0808 15:08:10.554261 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.554355 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:08:10.554749 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:41497: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab.
Entry for principal kudu/127.2.180.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab.
I20260808 15:08:10.582114 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.253:41497
--webserver_interface=127.2.180.253
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab
--principal=kudu/127.2.180.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:33111,127.2.180.253:41497,127.2.180.252:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:08:10.604056 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } attempt: 1
I20260808 15:08:10.605121 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.604650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42708)
0808 15:08:10.604841 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.604974 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 15:08:10.606050 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.605705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42724)
0808 15:08:10.605846 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.605922 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:08:10.606217 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:41497: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
I20260808 15:08:10.674494 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } attempt: 2
I20260808 15:08:10.675655 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.675140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42734)
0808 15:08:10.675373 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.675484 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 15:08:10.676672 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.676286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42746)
0808 15:08:10.676438 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.676521 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 15:08:10.676872 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:41497: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
I20260808 15:08:10.768154 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } attempt: 3
I20260808 15:08:10.769240 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.768805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42756)
0808 15:08:10.769002 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.769106 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260808 15:08:10.770206 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.769849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42764)
0808 15:08:10.769989 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.770058 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 15:08:10.770382 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:41497: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
W20260808 15:08:10.901891 4164 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:08:10.902220 4164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:08:10.902321 4164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:08:10.902438 4164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:08:10.909507 4164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:08:10.909602 4164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:08:10.909685 4164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:08:10.909750 4164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:08:10.909837 4164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:08:10.913604 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } attempt: 4
I20260808 15:08:10.914716 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.914178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42770)
0808 15:08:10.914493 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.914583 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146}
I20260808 15:08:10.915681 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:10.915330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42778)
0808 15:08:10.915472 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:10.915558 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:08:10.915901 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.253:41497: Network error: Client connection negotiation failed: client connection to 127.2.180.253:41497: connect: Connection refused (error 111)
I20260808 15:08:10.919860 4164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:33111,127.2.180.253:41497,127.2.180.252:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.253.keytab
--principal=kudu/127.2.180.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.253:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.2.180.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:08:10.921129 4164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:08:10.922441 4164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 15:08:10.933419 4171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:10.933477 4177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:10.933444 4173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:08:10.934506 4164 server_base.cc:1034] running on GCE node
I20260808 15:08:10.935238 4164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:08:10.936504 4164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:08:10.937731 4164 hybrid_clock.cc:648] HybridClock initialized: now 1786201690937657 us; error 34 us; skew 500 ppm
I20260808 15:08:10.938019 4164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:08:10 dist-test-slave-jlzn krb5kdc[4044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:08:10.941010 4164 init.cc:377] Logged in from keytab as kudu/127.2.180.253@KRBTEST.COM (short username kudu)
I20260808 15:08:10.942883 4164 webserver.cc:460] Webserver started at http://127.2.180.253:45311/ using document root <none> and password file <none>
I20260808 15:08:10.943500 4164 fs_manager.cc:359] Metadata directory not provided
I20260808 15:08:10.943601 4164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:08:10.943940 4164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:08:10.946650 4164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/data/instance:
uuid: "30f204c8fea54a478dc0723f6b5dd37a"
format_stamp: "Formatted at 2026-08-08 15:08:10 on dist-test-slave-jlzn"
server_key: "f7c124a12a66027d76836d32494cc151"
server_key_iv: "007eb30915320e2ce68dff60577eedb0"
server_key_version: "encryptionkey@0"
I20260808 15:08:10.947374 4164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/wal/instance:
uuid: "30f204c8fea54a478dc0723f6b5dd37a"
format_stamp: "Formatted at 2026-08-08 15:08:10 on dist-test-slave-jlzn"
server_key: "f7c124a12a66027d76836d32494cc151"
server_key_iv: "007eb30915320e2ce68dff60577eedb0"
server_key_version: "encryptionkey@0"
I20260808 15:08:10.952097 4164 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260808 15:08:10.955163 4190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:08:10.956378 4164 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 15:08:10.956542 4164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/wal
uuid: "30f204c8fea54a478dc0723f6b5dd37a"
format_stamp: "Formatted at 2026-08-08 15:08:10 on dist-test-slave-jlzn"
server_key: "f7c124a12a66027d76836d32494cc151"
server_key_iv: "007eb30915320e2ce68dff60577eedb0"
server_key_version: "encryptionkey@0"
I20260808 15:08:10.956725 4164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:08:10.979735 4164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:08:10.983196 4164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:08:10.983428 4164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:08:11.000022 4164 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.253:41497
I20260808 15:08:11.000067 4274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.253:41497 every 8 connection(s)
I20260808 15:08:11.001255 4164 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/data/info.pb
I20260808 15:08:11.004730 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 4164
I20260808 15:08:11.004881 4275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:08:11.004926 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-1/wal/instance
I20260808 15:08:11.005501 2771 external_mini_cluster.cc:1428] Setting key ddeb0e8b004c28575ca947186366eb7b
I20260808 15:08:11.013424 4275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:08:11.034474 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.016420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47738 (local address 127.2.180.254:33111)
0808 15:08:11.016632 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.016638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.017832 (+ 1194us) server_negotiation.cc:407] Connection header received
0808 15:08:11.020158 (+ 2326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.020164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.020252 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.020361 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.021750 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.022747 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.024276 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.024816 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.024934 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.028170 (+ 3236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.028177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.028181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.028222 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.031427 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.032177 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.032183 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.032186 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.032344 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.032732 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.032736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.032740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.033066 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.033232 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.033904 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.034132 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 15:08:11.035167 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.016873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:33111 (local address 127.0.0.1:47738)
0808 15:08:11.017650 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.017708 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:11.019860 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:11.020525 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:11.020531 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:11.021041 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:11.021552 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.021580 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.022897 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.022901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.024136 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.024149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.024996 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.025001 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.025194 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.025801 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:11.025823 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:11.027952 (+ 2129us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:11.031583 (+ 3631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.031589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.031600 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.032045 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.032435 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.032438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.032440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.032625 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.033169 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:11.033174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:11.033497 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 15:08:11.033751 (+ 254us) client_negotiation.cc:240] Negotiation successful
0808 15:08:11.034104 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":319,"threads_started":1}
I20260808 15:08:11.036824 4275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.2.180.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.2.180.252@KRBTEST.COM" created.
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:08:11.049958 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.037517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42790)
0808 15:08:11.037704 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.037723 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:11.037809 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:11.038819 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:11.038822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:11.039134 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:11.039267 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.039275 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.040695 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.040701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.041499 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.041510 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.042257 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.042263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.042444 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.042909 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:11.042925 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:11.044778 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:11.047656 (+ 2878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.047661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.047665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.048074 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.048417 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.048422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.048426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.048563 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.049102 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:11.049107 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:11.049240 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 15:08:11.049352 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 15:08:11.049595 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 15:08:11.050069 4280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.037755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.2.180.253:41497)
0808 15:08:11.038429 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.038436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.038466 (+ 30us) server_negotiation.cc:407] Connection header received
0808 15:08:11.038516 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.038524 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.038681 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.038788 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.039407 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.040573 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.041623 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.042126 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.042239 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.044961 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.044971 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.044983 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.045014 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.047533 (+ 2519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.048187 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.048193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.048195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.048313 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.048659 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.048664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.048667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.048974 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.049164 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.049480 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.049701 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":509,"thread_start_us":374,"threads_started":1}
I20260808 15:08:11.052289 4275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:08:11.053349 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.052882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49394)
0808 15:08:11.053043 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.053167 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 15:08:11.053580 4198 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
I20260808 15:08:11.055033 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.054692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49396)
0808 15:08:11.054835 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.054916 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 15:08:11.055258 4275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.2.180.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab.
Entry for principal kudu/127.2.180.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab.
I20260808 15:08:11.070475 2771 external_mini_cluster.cc:1326] Running /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
/tmp/dist-test-task6Pln5Y/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.252:44521
--webserver_interface=127.2.180.252
--webserver_port=0
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab
--principal=kudu/127.2.180.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.2.180.212:46493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.2.180.254:33111,127.2.180.253:41497,127.2.180.252:44521
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 15:08:11.104540 4275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 1
I20260808 15:08:11.105723 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.105245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49406)
0808 15:08:11.105448 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.105553 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 15:08:11.106734 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.106393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49410)
0808 15:08:11.106531 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.106624 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 15:08:11.106935 4275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
I20260808 15:08:11.175127 4275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 2
I20260808 15:08:11.176290 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.175844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49420)
0808 15:08:11.176036 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.176131 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 15:08:11.177218 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.176925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49436)
0808 15:08:11.177038 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.177106 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260808 15:08:11.177426 4275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
I20260808 15:08:11.215150 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } attempt: 5
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:08:11.229172 4280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.215984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.2.180.253:41497)
0808 15:08:11.216162 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.216167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.216192 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:08:11.216411 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.216415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.216493 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.216581 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.217613 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.218365 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.220017 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.220441 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.220530 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.224164 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.224168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.224172 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.224204 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.226660 (+ 2456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.227276 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.227281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.227283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.227403 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.227774 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.227777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.227779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.228103 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.228240 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.228658 (+ 418us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.228867 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 15:08:11.229167 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.215865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42794)
0808 15:08:11.216101 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.216127 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:11.216288 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:11.216648 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:11.216653 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:11.217207 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:11.217476 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.217492 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.218509 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.218515 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.219895 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.219913 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.220594 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.220601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.220861 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.221486 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:11.221510 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:11.223972 (+ 2462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:11.226784 (+ 2812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.226790 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.226795 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.227153 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.227506 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.227511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.227515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.227661 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.228220 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:11.228225 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:11.228376 (+ 151us) client_negotiation.cc:769] Sending connection context
0808 15:08:11.228516 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 15:08:11.228802 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 15:08:11.230103 4154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } has no permanent_uuid. Determining permanent_uuid...
I20260808 15:08:11.231079 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.230677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49438)
0808 15:08:11.230830 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.230914 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 15:08:11.232014 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.231651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49444)
0808 15:08:11.231832 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.231914 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 15:08:11.232251 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
I20260808 15:08:11.268620 4275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 3
I20260808 15:08:11.269742 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.269289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49454)
0808 15:08:11.269471 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.269579 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 15:08:11.270762 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.270334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49466)
0808 15:08:11.270556 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.270643 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
W20260808 15:08:11.270970 4275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
I20260808 15:08:11.283454 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 1
I20260808 15:08:11.284425 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.284018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49468)
0808 15:08:11.284194 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.284276 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 15:08:11.285341 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.285016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49484)
0808 15:08:11.285169 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.285237 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 15:08:11.285531 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
I20260808 15:08:11.353749 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 2
I20260808 15:08:11.354933 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.354462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49490)
0808 15:08:11.354683 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.354775 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 15:08:11.355903 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.355546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49502)
0808 15:08:11.355693 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.355762 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 15:08:11.356097 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
W20260808 15:08:11.391991 4283 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 15:08:11.392302 4283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 15:08:11.392398 4283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 15:08:11.392506 4283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 15:08:11.399485 4283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 15:08:11.399571 4283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 15:08:11.399672 4283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 15:08:11.399736 4283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 15:08:11.399833 4283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 15:08:11.409160 4283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.2.180.212:46493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.2.180.254:33111,127.2.180.253:41497,127.2.180.252:44521
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/krb5kdc/kudu_127.2.180.252.keytab
--principal=kudu/127.2.180.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.2.180.252:44521
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.2.180.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 13:58:48 UTC on ccycloud.quasar-qamcem.root.comops.site
build id 4574928
ASAN enabled
I20260808 15:08:11.410321 4283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 15:08:11.411589 4283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 15:08:11.414183 4275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 4
I20260808 15:08:11.415305 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.414837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49508)
0808 15:08:11.415055 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.415147 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 15:08:11.416344 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.415987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49516)
0808 15:08:11.416133 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.416210 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 15:08:11.416601 4275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
W20260808 15:08:11.422644 4292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:11.422676 4294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 15:08:11.422734 4298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 15:08:11.423697 4283 server_base.cc:1034] running on GCE node
I20260808 15:08:11.424482 4283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 15:08:11.425786 4283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 15:08:11.426987 4283 hybrid_clock.cc:648] HybridClock initialized: now 1786201691426913 us; error 46 us; skew 500 ppm
I20260808 15:08:11.427290 4283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 15:08:11.430310 4283 init.cc:377] Logged in from keytab as kudu/127.2.180.252@KRBTEST.COM (short username kudu)
I20260808 15:08:11.431704 4283 webserver.cc:460] Webserver started at http://127.2.180.252:34671/ using document root <none> and password file <none>
I20260808 15:08:11.432379 4283 fs_manager.cc:359] Metadata directory not provided
I20260808 15:08:11.432482 4283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 15:08:11.432793 4283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 15:08:11.435592 4283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/data/instance:
uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28"
format_stamp: "Formatted at 2026-08-08 15:08:11 on dist-test-slave-jlzn"
server_key: "2f262409df91c6c391de56c5b4cb9039"
server_key_iv: "9757acf9c74a185428594f05af604aeb"
server_key_version: "encryptionkey@0"
I20260808 15:08:11.436378 4283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/wal/instance:
uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28"
format_stamp: "Formatted at 2026-08-08 15:08:11 on dist-test-slave-jlzn"
server_key: "2f262409df91c6c391de56c5b4cb9039"
server_key_iv: "9757acf9c74a185428594f05af604aeb"
server_key_version: "encryptionkey@0"
I20260808 15:08:11.441054 4283 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260808 15:08:11.444206 4308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 15:08:11.445395 4283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 15:08:11.445552 4283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/data,/tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/wal
uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28"
format_stamp: "Formatted at 2026-08-08 15:08:11 on dist-test-slave-jlzn"
server_key: "2f262409df91c6c391de56c5b4cb9039"
server_key_iv: "9757acf9c74a185428594f05af604aeb"
server_key_version: "encryptionkey@0"
I20260808 15:08:11.445751 4283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 15:08:11.462297 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 3
I20260808 15:08:11.463245 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.462816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49522)
0808 15:08:11.462992 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.463077 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 15:08:11.464217 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.463882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49532)
0808 15:08:11.464040 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.464113 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 15:08:11.464399 4154 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.2.180.252:44521: Network error: Client connection negotiation failed: client connection to 127.2.180.252:44521: connect: Connection refused (error 111)
I20260808 15:08:11.466091 4283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 15:08:11.469894 4283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 15:08:11.470124 4283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 15:08:11.486711 4283 rpc_server.cc:307] RPC server started. Bound to: 127.2.180.252:44521
I20260808 15:08:11.486732 4385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.2.180.252:44521 every 8 connection(s)
I20260808 15:08:11.487973 4283 server_base.cc:1166] Dumped server information to /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/data/info.pb
I20260808 15:08:11.491412 4386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 15:08:11.494616 2771 external_mini_cluster.cc:1388] Started /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu as pid 4283
I20260808 15:08:11.494808 2771 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task6Pln5Y/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786201338129629-2771-0/minicluster-data/master-2/wal/instance
I20260808 15:08:11.495366 2771 external_mini_cluster.cc:1428] Setting key 050c0e23f5bbece9bbf47cef9ee1ba13
I20260808 15:08:11.498839 4386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
I20260808 15:08:11.516654 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.501678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47742 (local address 127.2.180.254:33111)
0808 15:08:11.501876 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.501882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.502646 (+ 764us) server_negotiation.cc:407] Connection header received
0808 15:08:11.502825 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.502831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.502905 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.503005 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.504280 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.505079 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.506692 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.507118 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.507208 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.510467 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.510474 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.510479 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.510513 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.513473 (+ 2960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.514213 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.514218 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.514220 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.514390 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.514877 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.514881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.514884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.515198 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.515365 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.516008 (+ 643us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.516245 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 15:08:11.517556 4390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.501752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.254:33111 (local address 127.0.0.1:47742)
0808 15:08:11.502504 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.502550 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:11.502702 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:11.503041 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:11.503045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:11.503799 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:11.504138 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.504150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.505243 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.505247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.506554 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.506569 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.507334 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.507343 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.507523 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.508105 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:11.508134 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:11.510289 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:11.513580 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.513590 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.513600 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.514121 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.514506 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.514510 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.514513 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.514728 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.515353 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:11.515360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:11.515549 (+ 189us) client_negotiation.cc:769] Sending connection context
0808 15:08:11.515874 (+ 325us) client_negotiation.cc:240] Negotiation successful
0808 15:08:11.516298 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":349,"threads_started":1}
I20260808 15:08:11.517576 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.497145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49542 (local address 127.2.180.252:44521)
0808 15:08:11.497795 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.497807 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.497846 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:08:11.499399 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.499421 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.499796 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.500174 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.501678 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.503276 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.504810 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.505617 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.505743 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.510157 (+ 4414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.510164 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.510174 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.510203 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.513445 (+ 3242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.514226 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.514233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.514236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.514413 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.514806 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.514811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.514814 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.515442 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.515723 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.516715 (+ 992us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.516937 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":279,"threads_started":1}
I20260808 15:08:11.519239 4386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.254@KRBTEST.COM
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
I20260808 15:08:11.533928 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.520361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47754 (local address 127.2.180.254:33111)
0808 15:08:11.520498 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.520503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.520518 (+ 15us) server_negotiation.cc:407] Connection header received
0808 15:08:11.520634 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.520639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.520698 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.520781 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.521774 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.522852 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.524018 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.524613 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.524734 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.529036 (+ 4302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.529042 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.529047 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.529080 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.531659 (+ 2579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.532276 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.532281 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.532285 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.532422 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.532793 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.532795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.532797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.533031 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.533307 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.533558 (+ 251us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.533710 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 15:08:11.537602 4390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.520023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.253:41497 (local address 127.0.0.1:42808)
0808 15:08:11.524622 (+ 4599us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.524640 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:11.524738 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:11.525063 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:11.525068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:11.525458 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:11.525600 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.525608 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.526903 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.526907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.527985 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.528000 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.528675 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.528681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.528857 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.529298 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:11.529319 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:11.531362 (+ 2043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:11.535187 (+ 3825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.535192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.535197 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.535646 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.536091 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.536094 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.536096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.536231 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.536763 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:11.536767 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:11.536905 (+ 138us) client_negotiation.cc:769] Sending connection context
0808 15:08:11.537019 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 15:08:11.537264 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4482}
I20260808 15:08:11.537737 4280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.520050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42808 (local address 127.2.180.253:41497)
0808 15:08:11.520249 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.520255 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.524709 (+ 4454us) server_negotiation.cc:407] Connection header received
0808 15:08:11.524861 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.524867 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.524946 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.525046 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.525742 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.526752 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.528124 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.528540 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.528658 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.531563 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.531569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.531574 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.531613 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.535072 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.535765 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.535770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.535773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.535979 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.536345 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.536351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.536358 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.536647 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.536775 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.537133 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.537330 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 15:08:11.538614 4386 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.2.180.253@KRBTEST.COM
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201691, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.252@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
I20260808 15:08:11.550037 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.535462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42812 (local address 127.2.180.253:41497)
0808 15:08:11.536054 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.536059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.536081 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:08:11.536122 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.536125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.536175 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.536255 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.537408 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.538347 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.539851 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.540459 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.540569 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.544886 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.544893 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.544897 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.544929 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.547491 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.548117 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.548122 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.548126 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.548250 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.548617 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.548622 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.548625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.548963 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.549226 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.549441 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.549642 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":228,"threads_started":1}
I20260808 15:08:11.552443 4390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.539195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49556)
0808 15:08:11.539333 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.539350 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:11.539434 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:11.539786 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:11.539790 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:11.540254 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:11.540415 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.540426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.541598 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.541602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.542421 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.542431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.543306 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.543311 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.543558 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.544896 (+ 1338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:11.544917 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:11.547028 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:11.550086 (+ 3058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.550091 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.550095 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.550477 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.550862 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.550867 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.550870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.551011 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.551542 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:11.551547 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:11.551675 (+ 128us) client_negotiation.cc:769] Sending connection context
0808 15:08:11.551829 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 15:08:11.552077 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":5}
I20260808 15:08:11.552505 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.539376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.2.180.252:44521)
0808 15:08:11.539532 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.539538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.539556 (+ 18us) server_negotiation.cc:407] Connection header received
0808 15:08:11.539599 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.539604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.539676 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.539775 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.540551 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.541475 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.542557 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.543166 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.543295 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.547196 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.547201 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.547204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.547230 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.549934 (+ 2704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.550586 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.550591 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.550595 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.550745 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.551115 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.551120 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.551123 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.551422 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.551558 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.551930 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.552231 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":49}
I20260808 15:08:11.554299 2771 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 15:08:11.557821 4386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: Bootstrap starting.
I20260808 15:08:11.557179 4121 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fa06706173f34959ae1186390853ffd9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47754
I20260808 15:08:11.560848 4386 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: Neither blocks nor log segments found. Creating new log.
I20260808 15:08:11.561861 4386 log.cc:824] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: Log is configured to *not* fsync() on all Append() calls
I20260808 15:08:11.564417 4386 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: No bootstrap required, opened a new log
I20260808 15:08:11.571045 4386 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } }
I20260808 15:08:11.571322 4386 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ebee8e7b4fe4c8c8128e8c3f8b1bb28, State: Initialized, Role: FOLLOWER
I20260808 15:08:11.571727 4386 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } }
I20260808 15:08:11.572600 4394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.572985 4394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:11.573509 4386 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 15:08:11.582265 4413 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 15:08:11.582392 4413 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 15:08:11.580003 4283 master_runner.cc:186] Error getting master registration for 127.2.180.254:33111: OK, instance_id { permanent_uuid: "fa06706173f34959ae1186390853ffd9" instance_seqno: 1786201690510708 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 15:08:11.587828 4283 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 15:08:11.641587 4154 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 4
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.254@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
I20260808 15:08:11.655102 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.642362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:49562)
0808 15:08:11.642572 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.642594 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:11.642729 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:11.643098 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:11.643103 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:11.643505 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:11.643672 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.643682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.644762 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.644768 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.645712 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.645726 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.646498 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.646504 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.646713 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.647276 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:11.647295 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:11.649597 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:11.652682 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.652688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.652692 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.653045 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.653399 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.653404 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.653408 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.653541 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.654126 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:11.654131 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:11.654289 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 15:08:11.654405 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 15:08:11.654699 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260808 15:08:11.655112 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.642413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.2.180.252:44521)
0808 15:08:11.642626 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.642631 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.642653 (+ 22us) server_negotiation.cc:407] Connection header received
0808 15:08:11.642876 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.642882 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.642953 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.643035 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.643848 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.644615 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.645861 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.646343 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.646459 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.649776 (+ 3317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.649783 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.649788 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.649821 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.652551 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.653165 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.653170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.653174 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.653293 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.653644 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.653649 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.653653 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.653992 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.654140 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.654535 (+ 395us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.654747 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 15:08:11.659945 4154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9: Bootstrap starting.
I20260808 15:08:11.663849 4154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9: Neither blocks nor log segments found. Creating new log.
I20260808 15:08:11.665069 4154 log.cc:824] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9: Log is configured to *not* fsync() on all Append() calls
I20260808 15:08:11.668198 4154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9: No bootstrap required, opened a new log
I20260808 15:08:11.675486 4154 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } }
I20260808 15:08:11.675915 4154 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa06706173f34959ae1186390853ffd9, State: Initialized, Role: FOLLOWER
I20260808 15:08:11.676394 4154 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } }
I20260808 15:08:11.677584 4417 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.678172 4417 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:11.678639 4154 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:08:11.678697 4121 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 15:08:11.678869 4121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:08:11.682220 4121 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } }
I20260808 15:08:11.684494 4121 leader_election.cc:288] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [CANDIDATE]: Term 1 election: Requested vote from peers 30f204c8fea54a478dc0723f6b5dd37a (127.2.180.253:41497), 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 (127.2.180.252:44521)
I20260808 15:08:11.684834 4351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa06706173f34959ae1186390853ffd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28"
I20260808 15:08:11.685367 4351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:08:11.689029 4351 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa06706173f34959ae1186390853ffd9 in term 1.
I20260808 15:08:11.689675 4086 leader_election.cc:302] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ebee8e7b4fe4c8c8128e8c3f8b1bb28, fa06706173f34959ae1186390853ffd9; no voters:
I20260808 15:08:11.691277 4417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 15:08:11.692322 4417 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [term 1 LEADER]: Becoming Leader. State: Replica: fa06706173f34959ae1186390853ffd9, State: Running, Role: LEADER
I20260808 15:08:11.692873 4417 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } }
I20260808 15:08:11.701539 4432 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa06706173f34959ae1186390853ffd9. Latest consensus state: current_term: 1 leader_uuid: "fa06706173f34959ae1186390853ffd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.702045 4432 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:11.715778 4275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } attempt: 5
W20260808 15:08:11.722018 4316 tablet.cc:2369] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:08:11.731011 4449 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 15:08:11.731740 4438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 15:08:11.744259 4389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.712730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.2.180.252:44521)
0808 15:08:11.712911 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.712917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.728368 (+ 15451us) server_negotiation.cc:407] Connection header received
0808 15:08:11.728528 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.728533 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.728614 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.728719 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.730458 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.731490 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.733415 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.734081 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.734194 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.737206 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.737212 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.737217 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.737255 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.740698 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.741535 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.741540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.741544 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.741696 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.742169 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.742174 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.742177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.742501 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.742722 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.743661 (+ 939us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.743900 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 15:08:11.737385 4063 master_runner.cc:186] Error getting master registration for 127.2.180.253:41497: OK, instance_id { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" instance_seqno: 1786201690995311 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
W20260808 15:08:11.750856 4089 tablet.cc:2369] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 15:08:11.751406 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.708586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.2.180.253:41497)
0808 15:08:11.708824 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.708832 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.712629 (+ 3797us) server_negotiation.cc:407] Connection header received
0808 15:08:11.713546 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.713553 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.713656 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.713728 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.718405 (+ 4677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.719756 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.725271 (+ 5515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.725945 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.726043 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.732087 (+ 6044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.732094 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.732099 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.732137 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.736063 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.748537 (+ 12474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.748543 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.748546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.748707 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.749164 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.749169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.749172 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.749543 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.749780 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.750861 (+ 1081us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.751100 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:08:11.751778 4063 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 08 15:08:11 dist-test-slave-jlzn krb5kdc[4044](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786201690, etypes {rep=17 tkt=17 ses=17}, kudu/127.2.180.253@KRBTEST.COM for kudu/127.2.180.252@KRBTEST.COM
I20260808 15:08:11.759151 4438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 15:08:11.774200 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.709566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54788 (local address 127.2.180.254:33111)
0808 15:08:11.709728 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.709733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.717704 (+ 7971us) server_negotiation.cc:407] Connection header received
0808 15:08:11.717854 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.717859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.717934 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.718025 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.739509 (+ 21484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.740719 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.747926 (+ 7207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.748683 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.748795 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.761797 (+ 13002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.761804 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.761809 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.761849 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.765559 (+ 3710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.771423 (+ 5864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.771428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.771432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.771679 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.772173 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.772179 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.772182 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.772617 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.772865 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.773461 (+ 596us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.773778 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 15:08:11.776682 4351 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [term 1 FOLLOWER]: Refusing update from remote peer fa06706173f34959ae1186390853ffd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:08:11.777706 4432 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:08:11.778445 4278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.716325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.2.180.252:44521 (local address 127.0.0.1:37332)
0808 15:08:11.716664 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 15:08:11.716682 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 15:08:11.716784 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 15:08:11.736941 (+ 20157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 15:08:11.736946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 15:08:11.737436 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 15:08:11.737657 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.737669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.743888 (+ 6219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.743894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.745259 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 15:08:11.745275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.751796 (+ 6521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.751822 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 15:08:11.752079 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.752710 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 15:08:11.752734 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 15:08:11.755345 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 15:08:11.774912 (+ 19567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.774918 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.774922 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.775445 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.776534 (+ 1089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 15:08:11.776540 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 15:08:11.776544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 15:08:11.776717 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 15:08:11.777435 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 15:08:11.777440 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 15:08:11.777603 (+ 163us) client_negotiation.cc:769] Sending connection context
0808 15:08:11.777750 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 15:08:11.778039 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
I20260808 15:08:11.783664 4447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:11.716989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37332 (local address 127.2.180.252:44521)
0808 15:08:11.736532 (+ 19543us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:11.736537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:11.736562 (+ 25us) server_negotiation.cc:407] Connection header received
0808 15:08:11.736606 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:11.736611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:11.736675 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:11.737812 (+ 1137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:11.738054 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.739320 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.747289 (+ 7969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:11.749906 (+ 2617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:11.754123 (+ 4217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:11.755603 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 15:08:11.755610 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 15:08:11.755615 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 15:08:11.755651 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 15:08:11.768079 (+ 12428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.775923 (+ 7844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.775929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.775933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.776096 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 15:08:11.776933 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 15:08:11.776937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 15:08:11.776940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 15:08:11.777293 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 15:08:11.779101 (+ 1808us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:11.779222 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 15:08:11.779427 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19394,"thread_start_us":19774,"threads_started":1}
I20260808 15:08:11.791080 4275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a: Bootstrap starting.
I20260808 15:08:11.795032 4394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa06706173f34959ae1186390853ffd9. Latest consensus state: current_term: 1 leader_uuid: "fa06706173f34959ae1186390853ffd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.795327 4394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:11.801333 4454 mvcc.cc:205] Tried to move back new op lower bound from 7316282129505955840 to 7316282129187180544. Current Snapshot: MvccSnapshot[applied={T|T < 7316282129505955840}]
I20260808 15:08:11.802361 4275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a: Neither blocks nor log segments found. Creating new log.
I20260808 15:08:11.804208 4451 mvcc.cc:205] Tried to move back new op lower bound from 7316282129505955840 to 7316282129187180544. Current Snapshot: MvccSnapshot[applied={T|T < 7316282129505955840}]
I20260808 15:08:11.804406 4275 log.cc:824] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a: Log is configured to *not* fsync() on all Append() calls
I20260808 15:08:11.816609 4275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a: No bootstrap required, opened a new log
I20260808 15:08:11.827181 4394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa06706173f34959ae1186390853ffd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.827456 4394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:11.830482 4275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } }
I20260808 15:08:11.830981 4275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30f204c8fea54a478dc0723f6b5dd37a, State: Initialized, Role: FOLLOWER
I20260808 15:08:11.831560 4275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } }
I20260808 15:08:11.832350 4417 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa06706173f34959ae1186390853ffd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.832645 4417 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:11.833411 4438 catalog_manager.cc:1347] Generated new cluster ID: dd9987d0ab4e4f6da73acfc9440756b0
I20260808 15:08:11.833534 4438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 15:08:11.863590 4275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 15:08:11.863688 4238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa06706173f34959ae1186390853ffd9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "30f204c8fea54a478dc0723f6b5dd37a"
I20260808 15:08:11.864530 4236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [term 0 FOLLOWER]: Advancing to term 1
I20260808 15:08:11.868403 4236 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [term 1 FOLLOWER]: Refusing update from remote peer fa06706173f34959ae1186390853ffd9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 15:08:11.872834 4238 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa06706173f34959ae1186390853ffd9 in term 1.
I20260808 15:08:11.873983 4459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.874709 4417 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 15:08:11.874820 4459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:11.885950 4438 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 15:08:11.887969 4438 catalog_manager.cc:1504] Loading token signing keys...
I20260808 15:08:11.905404 4438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9: Generated new TSK 0
I20260808 15:08:11.906342 4438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 15:08:11.924628 4459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa06706173f34959ae1186390853ffd9. Latest consensus state: current_term: 1 leader_uuid: "fa06706173f34959ae1186390853ffd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.924935 4459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:11.925354 4466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa06706173f34959ae1186390853ffd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.925747 4466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a [sys.catalog]: This master's current role is: FOLLOWER
I20260808 15:08:11.926738 4432 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa06706173f34959ae1186390853ffd9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa06706173f34959ae1186390853ffd9" member_type: VOTER last_known_addr { host: "127.2.180.254" port: 33111 } } peers { permanent_uuid: "30f204c8fea54a478dc0723f6b5dd37a" member_type: VOTER last_known_addr { host: "127.2.180.253" port: 41497 } } peers { permanent_uuid: "3ebee8e7b4fe4c8c8128e8c3f8b1bb28" member_type: VOTER last_known_addr { host: "127.2.180.252" port: 44521 } } }
I20260808 15:08:11.926996 4432 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa06706173f34959ae1186390853ffd9 [sys.catalog]: This master's current role is: LEADER
I20260808 15:08:11.980409 4485 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260808 15:08:12.032707 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.030306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44408 (local address 127.2.180.253:41497)
0808 15:08:12.030517 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.030524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.030693 (+ 169us) server_negotiation.cc:407] Connection header received
0808 15:08:12.030960 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.030966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.031045 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.031114 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.032453 (+ 1339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44408: BlockingRecv error: recv got EOF from 127.0.0.1:44408 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 15:08:12.034874 4447 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.031360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37336 (local address 127.2.180.252:44521)
0808 15:08:12.031559 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.031565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.033350 (+ 1785us) server_negotiation.cc:407] Connection header received
0808 15:08:12.033596 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.033601 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.033709 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.033782 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.034645 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37336: BlockingRecv error: recv got EOF from 127.0.0.1:37336 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 15:08:12.038616 4157 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.036621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54792 (local address 127.2.180.254:33111)
0808 15:08:12.036864 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.036870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.037168 (+ 298us) server_negotiation.cc:407] Connection header received
0808 15:08:12.037348 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.037353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.037434 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.037533 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.038391 (+ 858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54792: BlockingRecv error: recv got EOF from 127.0.0.1:54792 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 15:08:12.049638 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.042271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44420 (local address 127.2.180.253:41497)
0808 15:08:12.042422 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.042427 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.048305 (+ 5878us) server_negotiation.cc:407] Connection header received
0808 15:08:12.048472 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.048477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.048608 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.048701 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.049440 (+ 739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44420: BlockingRecv error: recv got EOF from 127.0.0.1:44420 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 15:08:12.055174 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.053869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44426 (local address 127.2.180.253:41497)
0808 15:08:12.054022 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.054027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.054043 (+ 16us) server_negotiation.cc:407] Connection header received
0808 15:08:12.054110 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.054114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.054188 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.054286 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.054967 (+ 681us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44426: BlockingRecv error: recv got EOF from 127.0.0.1:44426 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 15:08:12.065578 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.060744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44442 (local address 127.2.180.253:41497)
0808 15:08:12.060913 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.060920 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.064323 (+ 3403us) server_negotiation.cc:407] Connection header received
0808 15:08:12.064479 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.064484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.064562 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.064669 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.065400 (+ 731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44442: BlockingRecv error: recv got EOF from 127.0.0.1:44442 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 15:08:12.077077 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.074995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44450 (local address 127.2.180.253:41497)
0808 15:08:12.075214 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.075220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.075700 (+ 480us) server_negotiation.cc:407] Connection header received
0808 15:08:12.075873 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.075878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.075969 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.076072 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.076850 (+ 778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44450: BlockingRecv error: recv got EOF from 127.0.0.1:44450 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
W20260808 15:08:12.089864 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.088004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44466 (local address 127.2.180.253:41497)
0808 15:08:12.088154 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.088159 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.088635 (+ 476us) server_negotiation.cc:407] Connection header received
0808 15:08:12.088797 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.088803 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.088882 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.088970 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.089683 (+ 713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44466: BlockingRecv error: recv got EOF from 127.0.0.1:44466 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260808 15:08:12.110296 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.108728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44468 (local address 127.2.180.253:41497)
0808 15:08:12.108904 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.108909 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.109164 (+ 255us) server_negotiation.cc:407] Connection header received
0808 15:08:12.109300 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.109305 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.109398 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.109503 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.110060 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44468: BlockingRecv error: recv got EOF from 127.0.0.1:44468 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 15:08:12.151639 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.149555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.2.180.253:41497)
0808 15:08:12.149786 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.149792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.150197 (+ 405us) server_negotiation.cc:407] Connection header received
0808 15:08:12.150475 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.150479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.150549 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.150628 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.151402 (+ 774us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44476: BlockingRecv error: recv got EOF from 127.0.0.1:44476 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 15:08:12.221145 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.219384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44492 (local address 127.2.180.253:41497)
0808 15:08:12.219573 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.219579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.220016 (+ 437us) server_negotiation.cc:407] Connection header received
0808 15:08:12.220164 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.220168 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.220241 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.220327 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.220964 (+ 637us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44492: BlockingRecv error: recv got EOF from 127.0.0.1:44492 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 15:08:12.358987 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.357085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44504 (local address 127.2.180.253:41497)
0808 15:08:12.357312 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.357319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.357598 (+ 279us) server_negotiation.cc:407] Connection header received
0808 15:08:12.357799 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.357803 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.357888 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.357976 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.358756 (+ 780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44504: BlockingRecv error: recv got EOF from 127.0.0.1:44504 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 15:08:12.584424 4413 catalog_manager.cc:1259] Loaded cluster ID: dd9987d0ab4e4f6da73acfc9440756b0
I20260808 15:08:12.584527 4413 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: loading cluster ID for follower catalog manager: success
I20260808 15:08:12.586998 4413 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: acquiring CA information for follower catalog manager: success
I20260808 15:08:12.588791 4413 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ebee8e7b4fe4c8c8128e8c3f8b1bb28: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 15:08:12.620831 4392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 15:08:12.618724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44508 (local address 127.2.180.253:41497)
0808 15:08:12.618948 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:12.618955 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:12.619327 (+ 372us) server_negotiation.cc:407] Connection header received
0808 15:08:12.619535 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:12.619541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:12.619649 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:12.619773 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 15:08:12.620577 (+ 804us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44508: BlockingRecv error: recv got EOF from 127.0.0.1:44508 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 15:08:12.988894 4485 catalog_manager.cc:1259] Loaded cluster ID: dd9987d0ab4e4f6da73acfc9440756b0
I20260808 15:08:12.989040 4485 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a: loading cluster ID for follower catalog manager: success
I20260808 15:08:12.992769 4485 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a: acquiring CA information for follower catalog manager: success
I20260808 15:08:12.995179 4485 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 30f204c8fea54a478dc0723f6b5dd37a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 15:08:13.143600 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:13.136715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44516 (local address 127.2.180.253:41497)
0808 15:08:13.137380 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:13.137389 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:13.137652 (+ 263us) server_negotiation.cc:407] Connection header received
0808 15:08:13.137849 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:13.137853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:13.138015 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:13.138121 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:13.138638 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.139421 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.141355 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.141800 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.141893 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:13.142437 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:13.142997 (+ 560us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:13.143106 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:13.143240 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 15:08:13.143311 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":526,"thread_start_us":300,"threads_started":1}
I20260808 15:08:13.155453 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:13.149758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37352 (local address 127.2.180.252:44521)
0808 15:08:13.150573 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:13.150577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:13.150591 (+ 14us) server_negotiation.cc:407] Connection header received
0808 15:08:13.150689 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:13.150693 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:13.150830 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:13.150972 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:13.151383 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.152147 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.153388 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.153789 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.153891 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:13.154382 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:13.154861 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:13.155001 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:13.155093 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 15:08:13.155176 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":624,"threads_started":1}
I20260808 15:08:13.180372 4593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:13.161274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54804 (local address 127.2.180.254:33111)
0808 15:08:13.171482 (+ 10208us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:13.171489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:13.171528 (+ 39us) server_negotiation.cc:407] Connection header received
0808 15:08:13.171669 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:13.171675 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:13.172026 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:13.172225 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:13.172971 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.174302 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.176702 (+ 2400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.177384 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.177500 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:13.178253 (+ 753us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:13.178834 (+ 581us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:13.178931 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:13.179507 (+ 576us) server_negotiation.cc:299] Negotiation successful
0808 15:08:13.180018 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10067,"thread_start_us":10541,"threads_started":1}
I20260808 15:08:13.185513 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4063
I20260808 15:08:13.186455 4148 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:08:13.186714 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:13.168277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37364 (local address 127.2.180.252:44521)
0808 15:08:13.168424 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:13.168429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:13.180366 (+ 11937us) server_negotiation.cc:407] Connection header received
0808 15:08:13.180638 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:13.180644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:13.180745 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:13.180804 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:13.181410 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.182444 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.184515 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.185176 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.185275 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:13.185900 (+ 625us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:13.186045 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:13.186129 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:13.186339 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 15:08:13.186422 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 15:08:13.189357 4578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 15:08:13.161626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.2.180.253:41497)
0808 15:08:13.161800 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 15:08:13.161806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 15:08:13.179748 (+ 17942us) server_negotiation.cc:407] Connection header received
0808 15:08:13.180637 (+ 889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 15:08:13.180643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 15:08:13.180733 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 15:08:13.180795 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 15:08:13.181962 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.182739 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.186961 (+ 4222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 15:08:13.187663 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 15:08:13.187772 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 15:08:13.188434 (+ 662us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 15:08:13.188585 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 15:08:13.188651 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 15:08:13.188931 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 15:08:13.189029 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 15:08:13.298475 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 4063
I20260808 15:08:13.334844 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4164
I20260808 15:08:13.336017 4268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:08:13.446532 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 4164
I20260808 15:08:13.469872 2771 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu pid 4283
I20260808 15:08:13.470964 4381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 15:08:13.574533 2771 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task6Pln5Y/build/asan/bin/kudu with pid 4283
2026-08-08T15:08:13Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4933 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9953 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (355407 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST